2008 December 17 UT
Doug moved both red and blue hubs to their 'winter' positions, 1.25 mm away from primary. Note that prior to moving these, the red hub was at one limit; he turned the wrench both ways and there was no movement.
Note by JMH: We tried to use the above information to recover a lost LBC-Red hub position in January 2013. The location of the summer position was far away from what the above log entry implies.
Fernando mounted the PCI card in the Blue Science PC.
Fernando leak-tested the cryostat in the afternoon; it was good. He filled it with LN2 around 6 pm.
We turned LBC-Blue on with telescope at zenith. Success. We are moving to horizon to again to turn on the Blue camera. Then we'll try some OBs at a range of positions to test reliability. All closed-dome. Initial problem with connecting to LBT - so far this is not working, although the TCS is up and running. Aaron had not authorized red, just blue. When he authorized both blue and red, we could connect to the TCS. Don't we want the flexibility to authorize blue and red independently?
slew to galadi field at HA=01:23, elev=72 and take 'fake' data: lbcb.20081217.032754... cycles through 6 filters and 4 dither positions.
repeat galadi field
slew to SA95 and take this standard star data: 4 filters and 4 dither positions. UsUBV
filters. Checked data path: images are arriving in /Repository
25Bias_Bino.ob started. Elev=55 deg.
Turning off and on camera at elev=57 deg. Success.
2Darks_Bino lbcb.20081217.051130.fits, 052706 are both 15-min darks. Median dark cnts: 693, 787, 728, 701 and 674, 778, 718, 687 respectively.
Biases taken previously (lbcb.20081217.045*.fits) are 697-701, 777-779, 697-700, 687-690
Slew & 'take' another standard, GD50. Elevation = 54 degrees.
Take 2 more darks, this time 30-min each. Decided to stop the OB after 1. Median dark counts: 652, 765, 727, 683. Elev=53 degrees
A few biases: 671-664, 756-748, 709-698, 672-664. Elev=49 degrees.
Run StoneC_UBVVRI.ob with 3x exposure time (=60 sec) to test blue trackers, also slew, dither, change filters. Elevation 48deg.
OB execution failed with a RPC Client/Host communication error
Camera and shutter in error state. Try to run OB again without any other measures, but after ~3 minutes of doing nothing, it fails again with the same error as before (below is first occurence). 2 minutes later, OB ends with error.
2008/12/17 07:06:58.677229 E B CAMERA RPC Client/Host communication failed retval:5[src/camera/camera.c:1078]
2008/12/17 07:06:58.677306 S B CAMERA reinitialize camera [src/camera/camera.c:1081]
Turn off/on camera to see if this clears error. It does. Re-sent StoneC OB and it is working OK. As expected the tracker exposure time steps throgh 8,16,32 sec - not finding anything.
Try another OB, this one at higher elevation (elev=66), NGC2264_UBVRIz.ob. This does 120, 60, 60 sec exposures at U, B, V, 5 dithers. This takes the first U image (075326) but fails with an SPC-A200 status register read fail
2008/12/17 07:57:15.290000 E B CAMERA CCDCTRL "SPC-A200 status register read fail" (code:38) controller error [ccdctrl.c:2207]
2008/12/17 07:57:15.290000 E B CAMERA CCDCTRL hardware problems [ccdctrl.c:1715]
2008/12/17 07:57:15.292024 E B CAMERA CCD Controller error errcode:8 [src/camera/camera.c:1185]
2008/12/17 07:57:15.292121 W B camera system encountered a problem
2008/12/17 07:57:15.292148 N B expose completed
2008/12/17 07:57:15.477000 E B CAMERA CCDCTRL "SPC-A200 status register read fail" (code:38) controller error [ccdctrl.c:2207]
Turn off/on camera to see if it clears error. No, blue camera & shutter did not come up. Trying again but again the camera & shutter do not come up. load LCA on SPC-A200 error (SPC-A200 status register read fail) [ccdctrl.c:824]
The 3rd power cycle of LBC-Blue did the trick. Re-playing the OB. Added a note to IssueTrak #839
Temperature (from SMT) dropped sharply since midnight - from -5 C to -6.5 C at 01:30. (Of course the dome is closed, and so probably if there was a drop in dome temperature it was not so sharp).
Another SPC-A200 register read fail error. This time on the 60s B exposure of the 3rd dither.
After 4 power cycles and still no success getting the camera & shutters back up, we are slewing to zenith to see if that helps. Elevation was 64 deg. At zenith, camera & shutter came up on the first try.
Back to NGC2264_UBVRIz to try to repeat the 5-dither sequence. Failed on 6th exposure with same error. This time the power cycle at position (elev ~60deg) worked on the first try. Replayed the sequence (I did not reload the OB, simply re-played it but it seemed to start from the beginning as expected - read dither 1/5 and coordinates/filter were consistent with that. Got the same SPC-A200 error on the first image. Turning off/on at position. Success.
Slew to a position near zenith and try a 3-filter, 5-dither sequence with again U(120s), B(60s) and V(60s). See if this completes near zenith, as the OB NGC2264 never completed without the SPC-A200 error; furthest it got was to take 5 images
. Slewed to elev ~75deg (misread ST and didn't get so close to zenith as expected) but OB failed on first try (09:46 UT, 02:46 MST)
Slew to elev~90 deg. Slew and stop because previous OB had ended in error state. However, the power control page now shows all systems 'ready', so just play OB. Message in lbc.log at 09:48:40 UT says "SPC not found - reboot please" so maybe this accounts for switch from error or warning to 'ready'? In any case data taking fails - "RPC Client/Host communication failed" at 09:52 UT.
2008/12/17 09:52:43.029433 E B CAMERA RPC Client/Host communication failed retval:5 src/camera/camera.c:1078]
2008/12/17 09:52:43.029514 S B CAMERA reinitialize camera [src/camera/camera.c:1081]
Power cycle LBC.
Try a standard star observation - so trackers are not used. These standard star OBs used many filters and dithers yet ran through without error at the start of the night. At 10:21:37, on first exposure of SA101_UsUBVrRIY.ob, fails with SPC-A200 error.
Going to shut down. Hard to understand what happened to start the communications errors. All was going well from about 20:30 to midnight. Then, was it temperature?, use of trackers? or elevation? that started the communications errors? How do we recover?
- 17 Dec 2008