MODS1 AGw Commissioning at LDG: 24 September 2010 UT
Observer: J. Hill (LBTO), R. Pogge (OSU), R. Stoll (OSU), M. Pedani (LBTO), B. Atwood (OSU)
Telescope Operator: DHuerta (LBTO)
Telescope Support: JLittle (LBTO)
Instrument Support: JMorris (LBTO)
Closed for humidity for first 60% of night, but still plenty of things to debug. The biggest problem was that the AzCams require a manual initialization exposure for each different readout configuration - otherwise your image comes out scrambled.
This is semi-tolerable for the WFS, but it means you can't take guide and acq exposures with the guider. We ran a temporary fiber from LLTH to ULTH so that we could read the MODS guider with the AGW2 guider PC with the python AZCAM. By 1AM we seem to have everything working for closed dome presets and fake guiding, but that was a false sense of security. We opened at 2:16 AM MST. The first preset gave us a star
, so the old LDG pointing model wasn't so bad. We did some manual collimation. Then we got guiding and acquisition offsets working
by changing the X-flips from -1 to +1. Next we got the offsets to the nominal WFS hotspot working.
We started working on WFS, but had loads of problems with AzCam sending us repeat copies of the same image.
We only got IDL to be able to take new images, by taking a manual AzCam image before each one. The conclusion is that
we must switch to the Python Software
on both Windows machines.
Warning: There is a fiber stretched from LLTH to ULTH. Be careful while walking on the telescope.
Note: The fiber was removed later in the daytime.
MODS software updates
Pogge fixed software bugs in the MODS TCS code that was returning odd HAs around the meridian (sometimes), and has
made changes to the MODS control panel GUI that should prevent last night's problems with corruption of the
entry boxes with telescope status info (the parser that would fill the display panel was also filling the entry boxes, some
old code that should have been removed that must have been restored and not noticed until now).
Replace AzCam Guider Power Supply
Mark Wagner brought up the LBT8 Power Supply box from Tucson today. The LBT1 power supply box on the MODS1 guide camera
ARC controller. Tests during the morning showed that if we powered the WFS camera's ARC controller with the Guide
camera's (suspect) power supply, we could get the same fault as stopped the Guide Cam last night. However, after
numerous power cycles the suspect power supply stopped failing. We decided to tag it as "suspect and unreliable" and
swapped in the LBT8 power supply. Tests showed that it worked without failure. This closed Issue 2866.
MODS hardware upgrades
More work on the MODS science CCD controllers to enable the LN2 dewar blow-off gas inline heaters. These should prevent
the formation of ice on the vent line during high humidity. The red camera dewar's "bog" heater was installed yesterday and
there was no condensation or icing on the Red dewar vent port. Blue had condensation. The blue bog heater was installed
today and appears to work. We'll know for sure tomorrow (still humid).
MODS Instrument clamps
We discovered in the afternoon that 4 of the 6 clamps that hold MODS onto the telescope had worked loose overnight to the point that the levers that rotate the T-head bolts had rotated out to a radial orientation. That's not good!
These bolts had been torqued for a second time by Bruce A. and Doug O. on Monday (200 ft-pounds). We rotated the levers and torqued the bolts at zenith. The levers are now held in place with cable ties, so they should not rotate to the release position even if the ball interfaces work loose. Then we tipped over to horizon after the other MODS upgrades were complete. We found that 5 of 6 bolts were not fully torqued after the horizon move. Jeff U. retorqued all 6 at horizon in the gravity-preload orientation. Jeff was groaning with an 18-inch wrench handle, so we estimate 200 ft-pounds.
Closed for humidity.
AGw debugging for Issues 2868 and 2872
Note: After some discussion we figured out that the problems from 05:50 yesterday were two issues. One is related to park active probe when an AGW isn't selected.
The second is related to the double issue solved below in the interface between GCS and the MODS AGW server.
02:30 Debugging GCS to MODS AGw server communication. This is Issue 2868. The problem appears to be a confusion between MODS AGw units 0,1,2.
Switching the unit number from 0 to 1 in GCSL didn't help. So Rick is rebuilding the MODS AGw server code and we got rid of the 0/1 problem. But now we are getting another error
(with the same message) at the same call (mods_initTrans).
Sep 24 02:54:58 tcs4 LBT_left_GCS: error reading transformations from MODS server 192.168.139.130 for AGW #0
Sep 24 02:54:58 tcs4 LBT_left_GCS: error: during homing of AGW unit OSU_MODS.
03:30 Torsten makes us a patch to GCS to change a boolean logic fault in handling error vs. success returns on the mods_initTrans() function. That got rid of the error.
This got the MODS left Authorization to proceed. Hooray!
Closed dome presets
03:58 Closed Dome guide preset - fails complaining of empty guide star list, although the telescope did slew there. Problem seems to have been gs name "none".
04:02 Another guide preset. - fails for lack of guide star !!!
04:05 Use AzCam
tool to make an exposure. This trick needs an IssueTrak
. Note: Any such trick is moot, after the conversion to Python AzCam the following day.
04:08 Set flag in lbt.conf for overlayed fake guide star. Stop and restart GCSL. GCSGUI says "SIMULATED" which is not the right message. It should say "SIMULATED STAR".
04:10 Another guide preset succeeds up to the point of multiple acquisition images and fake guiding.
04:19 Rick sees an error on the MODS side when the PauseGuiding command succeeds.
04:22 Another guide preset - succeeded, but GCSL failed to (thought interupted here).
Interlude here for taking LED flash images inside MODS.
04:59 More presets to check probe position. It seems generally OK.
Torsten created Issue 2873 for the case where GCS reports an error when you authorize with (away from?) LBC.
05:05 Preset correctly fails with guide star outside the patrol field.
05:30 Humidity rapidly drops to 70%, but then goes back up to 92% 10 minutes later. Sky is clear. Dome is dripping lots of water.
Problems with guider image
Manual images in /OldRepository/AGW_Data/20100924/
Acquisition image was black with a grid of white spots until we took a manual image with the AzCamTool
(logging into the AzCam
machine via VNC). After one manual image, the images in GCS appear normal.
05:59 stop and restart GCS
Taking a picture with the AzCam
interface on the Windows PC cures the problem for the Acq Image, but then the guider image is screwed up.
Taking a sub-array picture with AzCam
looks OK, but then the Acq images are screwed up again.
06:54 Restart GCS with connection to AGW2 guider.
07:01 Torsten sends manual guide sequence so we don't have to move the telescope.
This works with good guider images and acquisition images.
readGuideCam left -e 1200 guider00006
07:17 Torsten puts in a new GCS build with some other minor fixes.
We send Torsten to bed so he can get up in the morning to install the Python version of AzCam
on the MODS AzCam computers.
07:39 Humidity has dropped to 50% so there is still hope.
More closed dome presets with the guider running from AGW2.
Add ROI call to
This GCS build seems to have problems and always shows SFP as 0,0.
08:16 David opens the rear vent doors. Humidity is down to 44%.
Named the GCS we were using GCS.2010092303.
08:20 Roll back GCS build to GCS.2010092302 (which was made about the same time).
08:23 repeat the previous preset still went to 0,0
08:25 Roll back GCS to GCS.2010092301 which is the version from 03:49 UT.
Changed Park to -90, -200 athough it was already -92,-242 which is not what GCS is sending -1,419
08:30 Open the dome, T=7.8 degC D=-1.4 degC Wind light from south, sky clear.
08:33 This GCS.2010092301 is the one with the initTrans bug from before. Torsten didn't use the -p flag when he copied
it at 03:49.
08::35 Set GCS to GCS.2010092302
08:39 remove the artificial guide star from lbt.conf
08:40 Send a preset with the dome open
08:49 Preset to M5_0042 in ACQUIRE (need all the bright pointing stars installed in the MODS catalog)
Holy Cow, we got a star in the first acquire image. IE=-24 CA=+21
08:59 Stop GCS and selectAGW
09:00 Rick types Dec +15 but PCS receives +05 ?????? The MODS interface requires the DEC sign.
09:04 M5_0050 at EL=71 Adjust IE=-37 CA=+23
Put in ND1 filter to collimate
Eyeball collimation: Zglobal=0.5 Z7=+3000
09:12 Guide preset on M5_0050 (10 ms exposure and ND1)
Not running away, but not guiding either. GCS thinks the star is too saturated.
09:19 Guide preset on WT10_244, but still too bright to get stable guiding
09:23 Guide preset to ACT_02xx -- failed for bad polys.
09:27 Guide preset to ACT_02xx - again bad polys
Restart PCS (was it confused by coordinates uploaded by MODS from reading one of the
09:30 Guide preset to ACT_02xx - again bad polys
09:31 different star - again bad polys
but then we see that the proper motions are bad into PCS, the MODS interface was not converting
proper motions in mas/year (astronomer units) into radians (engineer units). Problem fixed in the
MODS IIF agent code, and no more problems.
: 03:29:52.105742291 PM CorrDec
09:37 WT10_xx typed in by hand without proper motions - preset worked w/o PCS errors.
Had wrong IE CA
09:44 Failed because guiding was not stable, But GCS didn't send any corrections to PCS.
changing various parameters in OSU_L.cfg
back to 150 pixel guide image
maxoffset to 5.54
not offsetting or guiding, but the blue circle is big!
GCSL.noTCS set to false
10:06 Now we have guiding that runs away in X.
guidingCorr_flipX from -1 to +1
10:11 preset - and now we are guiding correctly.
10:14 preset - proper motions are correct
Z7 +2000 and Zglobal reduced to 0.25 mm
for deliberately poor pointing.
Y is correct, but X is wrong in the acq offset.
pointingCorr_flip_x 1.0 float # should the SFP X coordinate be flipped before sending to PCS for initial pointing correction? -1.0 means flip
pointingCorr_flip_y 1.0 float # should the SFP Y coordinate be flipped before sending to PCS for initial pointing correction? -1.0 means flip
guidingCorr_flip_x 1.0 float # should the SFP X coordinate be flipped before sending to PCS for guide correction? -1.0 means flip
guidingCorr_flip_y 1.0 float # should the SFP Y coordinate be flipped before sending to PCS for guide correction? -1.0 means flip
1 arcsec image.
guidecam_hotspot_x 262.0 double # ccd x coordinate of pixel that represents center of light in WFS pupil
guidecam_hotspot_y 421.0 double # ccd y coordinate of pixel that represents center of light in WFS pupil
Stop and restart GCSL, selectAGW
10:34 guide preset first offset worked to center, but offset of probe to the hotspot failed
guidecam_center_x 270. double # center of guidecam to use for inital GS acquisition with MODS
guidecam_center_y 225.0 double # center of guidecam to use for inital GS acquisition with MODS
WFScam_offset_x -0.25 double # position offset in x mm to move star from center onto WFS pinhole (guidecam hotspot)
WFScam_offset_y -12.75 double # position offset in y mm to move star from center onto WFS pinhole (guidecam hotspot)
These numbers come from measurements of the artificial star in the lab!
Stop and restart GCSL, selectAGW
10:48 preset active -- The offset works!!!!!
But no light is getting into the WFS, so now we are scanning the probe back and forth
setWFSExpTime left 4000
but seems to be same WFS image over and over ????
11:14 restart WFS server
now we get light in the WFS, but hit M1 limits
Clear collimation at resend +5000 Z7
11:23 The problem is that WFS is sending corrections to mirror every few seconds even though it claims 39 sec per image.
Between 11:19:07 and 11:20:06 it looks like we processed the same image with different names 12 times.
Uncomment the ROI in OSU_L_WFScam.ini
didn't help, so undo it. and take manual AzCam
Now it has the correct pace, but the same image over and over.
Stop GCS, Stop server, Restart server, take AzCam
, restart GCSL, selectAGW
11:48 Guide Preset
collimate_multiple, 1 made wfsc00006
collimate_multiple, 1, /send made wfsc00007
collimate_multiple, 1, /send made wfsc00008
But we seem to be getting the same image again each time.
collimate_multiple, 1, /send made wfsc00009 (a different better image)
collimate_multiple, 1, /send made wfsc000010 (same as 0009)
Stop reporting centroids to PCS
Offsetting probe, taking AzCam
This lets us adjust where the spots are in WFS, and we had some 5AM confusion that X and Y are rotated on WFS compared to guider.
collimate_multiple, 1, /send made wfsc000011
12:18 collimate_multiple, 1, /send made wfsc000012 (-.176, -14.79 on the probe)
collimate_multiple, 1, /send made wfsc000013
collimate_multiple, 1, /send made wfsc000014 (-0.376, -14.789)
collimate_multiple, 1, /send made wfsc000015 (seems to be bad correction)
12:27 Clear C00
collimate_multiple, 1, /send made wfsc000016
collimate_multiple, 1, /send made wfsc000017
Send +2000 Z7
Send -4000 Z7 and run into limits
Close for impending dawn
WFScam_offset_x -0.37 double # position offset in x mm to move star from center onto WFS pinhole (guidecam hotspot)
WFScam_offset_y -14.7 double # position offset in y mm to move star from center onto WFS pinhole (guidecam hotspot)
Acquisition Sequence for MODS
This is John's understanding of the three image acquisition sequence for MODS that is presently installed in GCS.
- First image: Star nominally centered based on telescope pointing
- Second image: Adjust telescope pointing to put star actually in the center
- Third image: Move guide probe to offset the star into the hotspot (inital hotspot is in center, so probe won't move, later we dialed in provisional offsets computed relative to a provisional center)
- 23 Sep 2010