Checkout at Right Front Bent Gregorian - 09 October 2011 UT

Observer(s): J. Hill (LBTO), plus Lorenzo, Alfio, Carmelo at the beginning
Telescope Operator: S. Allanson (LBTO)
Telescope Support: R. Hansen (LBTO)
Instrument Support: D. Officer (LBTO)


It was sooooo much better than last night. We were on-sky all night, and many things were fixed (but not SX M2 UMAC).

We spend the first part of the night checking out the basic function of AGW2 and RFBG. And we did find a couple of significant problems to fix. We also exercised all of the adaptive optics software (BP11) that we could without having a science detector.

We spent the last half of the night making field aberration measurements and adjustments. The grain of salt here is that the telescope structure was rapidly changing temperature, so these adjustments may not apply to equilibrium conditions.


Armando sorted out the problems with setting the DX shell.
Torsten sorted out the problems with AGW1 cameras.
Paul fixed the argument list problem that was bothering LBC. That will be ready for tomorrow's BP13 testing.
SX M2 UMAC is still inoperable, so we will be DX side only tonight. Chris brought the UMAC laptop to the mountain.



00:40 UT Open before sunset to cool the chamber and for tourism at the family picnic. Sky is mostly clear, but not photometric. Wind is light, 2-4 m/sec. Humidity about 70%, but decreasing during the night.

01:00 UT Start mirror ventilation, but it is struggling to get to -1 degC as we are still on the summer chiller (winter chiller is under repair).

Authorize as IRTC@RFBG, NONE on LEFT.

Initial Pointing Star

01:37 ACQUIRE Preset to M5_0738 - failed for no AGW unit selected. Stop and restart GCSR. That didn't help - call Torsten (no answer) - call Tom.

In the meantime, we reboot agw2-azcamg and agw2-azcamw, stop and startAGW, restart GCSR again.

Oct  9 01:53:42 tcs4 LBT_right_GCS: wait done for handle block:0:1 0 0
Oct  9 01:53:47 tcs4 LBT_right_GCS: GCSR:setGuidestarList executed from IIF, handle 1
Oct  9 01:53:47 tcs4 LBT_right_GCS: gcs.command.setGuidestarList called and no AGW unit selected
Oct  9 01:53:47 tcs4 LBT_right_GCS: gcs.command.setGuidestarList.precondition.notmet: no AGW unit selected.
Oct  9 01:53:47 tcs4 LBT_right_GCS: GCSR:setGuidestarList execution time: 24 ms from IIF, handle 1
Oct  9 01:53:54 tcs4 LBT_left_GCS: AGWUnit: UMAC temp: 39.0 C, ambient temp: 15.0 C, humidity: 30.0 %

02:00 Stop and restart GCSR again, same error.

02:10 John found the problem: AIP_IRTC_R.cfg had the guider port set to 2401, but agw2-azcamg was listening for commands on port 2402. Changed guidecam_port to 2402 in AIP_IRTC_R.cfg. (The same error exists in BP12 configuration - not yet corrected.) Issue 3715 asks Torsten to archive this.

02:12 Restart GCSR and now the preset succeeds. Found very defocussed star at IE=-2 CA=20. But, this star wasn't bright enough to be the correct ID.

02:23 Found suitably bright star at IE=56 CA=45.

Collimation stars at various elevations

02:26 Preset to BS9177 to confirm pointing and collimate there. EL=65. Seeing is 1.1-1.2 arcsec on guider. Bias+sky is 280 ADU on the guide cam. Bias on the WFS cam is 275 ADU.

02:38 Preset to BS9162 to check things at lower elevation EL=38. Seeing is 1.5 down here with some moderate astigmatism.

02:46 Preset to AO814=HD209438 Rmag=7.78 with off-axis guide star. Lots of astigmatism as we move back to EL=70. WFS applies ~+8000 nm astiigmatism in 6 iterations. Seeing is ~1.4.

Adaptive Optics Checkout

03:00 AO guys report that they are detecting photons on all their cameras. Now we'll close the AO loop from the engineering GUI.

03:25 Send ACE preset to same target with IRAF

We get an ICE exception when we issue the RunAO command. (exact text in Steve's Night Report)

03:33 Stop and Restart IIF to see if we can clear the exception. Register from IRAF.

03:34 Resend the ACE preset from IRAF although it was still running through the IIF restart.

Now trying IDL observing commands as John and Carmelo together know enough.

03:45 Doug's observe command has lots of restricted DD variables. Turn restrictions off and resend. Same error as we got from IRAF.
      runao: closing the AO loop
% Compiled module: IRC_RUNAO.
===> Error in irc_runao
        err = irc - InfraRed Camera Client V 3.0
        err = Outgoing.cpp:478: Ice::UnknownLocalException:
        err = unknown local exception:
        err = /usr/include/Ice/BasicStream.h:551: Ice::UnmarshalOutOfBoundsException:
        err = protocol error: out of bounds during unmarshaling

03:48 Steve tries restarting IRS/IRC, but we get the same error - this time before IIF received a preset. We needed to issue an iif_register command in IDL before running the observe command.

Call Chris (no answer - still on the road) - Call Tom. This is Issue 3714.

04:20 Tom changes the pointer for irs back to an older version (from version 3.0 of 3-Oct to version 2.4 of Dec).

04:25 Send new adaptive preset from the IDL observe command. (after switching PA from 90 to 270) This time RunAO was issued successfully.

04:29 Got a stopAO command? Which safety check sent this? Carmelo says this was a skip frame. (The camera lens is near its low temperature operating limit.

04:33 Resend the preset from IDL

04:43 Steve sends an absolute RADEC offset from IRAF while the loop was closed.

04:47 John/Carmelo send a detxy offset from IDL observe: offset, COORD=detxy, X=-2.0, Y=2
This 2,2 arcsec offset took 12 seconds.

04:50 Carmelo, last of the AO guys goes off to bed.

05:02 The tip-tilt and low-order zernike offloads to M2 hexapod and M1 forces are working smoothly. We've built-up only 10 arcsec of tip-tilt offload in 15 minutes, so we seem to be following the collimation model correctly (given the rapidly cooling telescope temperature).

05:07 Saved 4000 CCD47 frames. The seeing about 1.5 arcsec on guider, so the corrected images are not spectacular (I don't have quantitative numbers) at short wavelength, but there is good correction improving the seeing.

05:18 Steve sends ACE preset from IRAF to same target (different PA). IRAF (newpoint) doesn't wait for an active optics cycle before issueing RunAO. RunAO succeeded, but the loop opened a few seconds later because of skip frame. Seeing is still about 1.5 arcsec, so not very promising for other activities like collecting field data.

05:23 resend the ACE preset, same result with skip frame.

05:28 ACQUIRE preset to tune up pointing. IE=??

05:31 ACE Preset to AO778 = G 27-43 R=12.7 gs=2 (including Az unwrap), RunAO failed because star centering procedure didn't converge (active optics hadn't run so there was significant defocus and astigmatism. The preset correctly dropped back to active mode after RunAO failed.

05:39 irc RunAO right -
Sun Oct  9 05:40:42.339 2011  RunAO alert for right IRTC: RETRY: source_acq error: Star centering procedure failed to converge after 5 iterations (-20003) WFSARB_STAR_NOTCONVERGED
Sun Oct  9 05:40:42.370 2011  RunAO alert for right IRTC: AcquireRef[57] failed - RETRY: source_acq error: Star centering procedure failed to converge after 5 iterations (-20003) WFSARB_STAR_NOTCONVERGED
Sun Oct  9 05:40:42.401 2011  RunAO command failed from right IRTC, rePoint 0, leftRePoint 0, rightRePoint 0

At this point John and Steve are out of AO knowledge, so we'll switch to doing something else.

Field Data on BS9107 in mediocre seeing (Set A)

Steve sends ACQUIRE and ACTIVE preset on BS9107. Seeing on guider is 1.5 arcsec at EL=44 deg. IE=75 CA=53

05:57 field_collect (on BS9107_field.list from the pop-up window), preset fails in a rotator fault. Steve restarts MCSPU which fixed the problem.

06:01 restart field_collect, but shell had RIP'd because of restarting MCSPU.

Stopped again for an ACQUIRE preset as pointing was off after that episode. Why? IE=83 CA=29

06:08 field_collect again. Seeing is 1.4-1.7 arcsec, so this isn't going to be very good data. Think of it as a sanity check, however the off-axis zernikes are fairly large (2000 nm rms).

The active optics corrections on M2 are Xact=2.37, Yact=4.017, Zact=-0.4, RX=-418 RY=248, but C-rings are in rapid temperature change, so it isn't useful to add these to global offsets. Lower C-ring is still 3.5 degC above ambient air at midnight.

07:05 Seeing improved briefly to 0.9 arcsec, but it didn't last very long.

07:12 field_collect crashed......
  => Running collimate_process: /Repository/GCSfiles/dailyimages//right_wfscimage000394.fits
% Attempt to subscript WFSC_SET with I is out of range.
% Execution halted at: COLLIMATE_GCS     116
%                      FIELD_COLLECT     331
%                      COLLIMATE_GCS     129
%                      FIELD_COLLECT     200
%                      COLLIMATE_GCS     129
%                      FIELD_COLLECT     200
%                      COLLIMATE_GCS     129
%                      FIELD_COLLECT     200
%                      $MAIN$          
% Program caused arithmetic error: Floating illegal operand

GStar wfsc #
0 319-331
23 332-334
0 335
28 336-338
0 339-341
36 343-345
0 346-348
33 359-361
0 362-363
8 364-366
0 367-368
19 369-371
0 372-373
42 374-376
0 377-379
39 380-382
0 383-385

First dataset ends with wfscimage000394. The data are in /OldRepository/AGW_Data/20111009 with further files in subdirectory field.

 => Field Data Results
      Motion of the secondary need to remove bi-nodal astigmatims
              From Astigmatism Measurement
          Tip/R_x =  16.0", Delta IE = -16.0"
         Tilt/R_y = -79.5", Delta CA =  79.5"
Plane Fit: Ax + By + C = z
A =       3.98731
B =      -3.55582
C =      -551.765
s =       124.059
              From Focus Measurement
          Tip/R_x =  53.3", Delta IE = -53.3"
         Tilt/R_y = -59.8", Delta CA =  59.8"

Setting the inital collimation and pointing in a different way

07:15 ACQUIRE preset to pointing star at EL=77.5

ptreset to find initial IE=+4.7 CA=0.0, then return to where we were. Then we laterally adjust the global offset of M2 to restore the pointing to the initial IE and CA.

The original global offsets on M2: Xglob=-0.611 Yglob=-1.662 Zglob=0.738 RX=78.3 Ry=336.4 which gave pointing at IE=83 CA=39 above.

07:28 Clear active optics.

What we got after lateral translation: Xglob=+1.350 Yglob=+1.900 Zglob=-0.200 RX=78.3 RY=336.4 at EL=79.6 and IE=4.7 CA=0.0

Warning: I don't think that DX M2 moved by 2 mm in each axis when it got remounted, so much of this signal may still be thermal distortion.

Warning: We also need to reprocess these images to make sure GCS has the correct off-axis pupil position. (The pupil position is clearly wrong on the GCSGUI.)

07:33 Active Preset back to BS9107 to collimate.

Now the active corrections at EL=64.8 are Xact=-1.577 Yact=-3.549 Zact=0.050 RX=369.8 RY=-164.3 (interestingly opposite to what we had above)

At EL=65.2 DX M2 Totals: Xtot=+1.951 Ytot=-4.576 Ztot=+2.417 RX=298.0 RY=781.6

Different Set of Field Data on BS9107 (Set B)

07:39 field_collect on BS9107 for a separate set of data in this new collimation condition. Seeing starts at 0.85.
Outside air is warming, so things are getting into better equilibrium.

Second data set starts with wfscimage000408.

GStar wfsc #
0 408
23 409-411
0 412-413
28 414-416
0 417-418
36 419-421
0 422-423
33 424-426
0 427-431
8 432-434
0 435-439
19 440-442
0 443-445
42 446-448
0 449-450
39 451-453
0 454-455
43 456-458
0 459-460
44 461-463
0 464-468
14 469-471
0 472
12 473-475
0 476-478
41 479-481
0 482-484
22 485-487
0 488-489
24 490-492
0 493-495
20 496-498
0 499-501

Night lunch is really luxurious with all the leftover picnic food. I think I saw Steve eat a cheeseburger garnished with chicken!

08:10 Toutside=+0.0 so mirror is only 2/3 degC warm.

08:29 preset fails....
Sun Oct  9 08:29:54.669 2011  Preset alert for right IRTC mode ACTIVE: guide loop didn''t stop within timeout.   (10 seconds)
Sun Oct  9 08:29:54.684 2011  Preset command failed for right IRTC mode ACTIVE
I think this means our night has been successful. We exercised the system long enough that we found a very obscure bug.

08:33 field_collect, start=11, but it didn't work. GCSR hung while moving the guide probe for the first preset.

08:42 Steve kills GCSR and restarts.

08:44 field_collect, start=11, now it works.

09:18 finished.

 => Field Data Results
      Motion of the secondary need to remove bi-nodal astigmatims
              From Astigmatism Measurement
          Tip/R_x = -37.6", Delta IE =  37.6"
         Tilt/R_y = -71.5", Delta CA =  71.5"
Plane Fit: Ax + By + C = z
A =       3.21408
B =      0.546887
C =      -303.284
s =       183.593
              From Focus Measurement
          Tip/R_x =  -8.2", Delta IE =   8.2"
         Tilt/R_y = -48.2", Delta CA =  48.2"

Pointing and Collimation Check

Steve is driving around the sky for a pointing and collimation check while John tries to reduce the field data with Doug's routines.

Make Field Aberration Adjustment

Tilt/R_y = -71.5", Delta CA = 71.5"

09:54 ACQUIRE preset to BS9107 - but we found the wrong star? We got the bright star in the BS9107 field.

10:01 ACQUIRE preset to M5 star

New RYglob is 336.4 - 71.5 = 264.9, CA=+79.5 but no star there. Both the sign and scaling of delta CA compared to delta RY are wrong in Doug's analysis. CA should be -15.

Third round of field data on BS9107 (Set C)

10:18 field_collect (starting at wfsc000536, but don't use the first 4 frames because of M2 astigmatism) Seeing starts at 0.7 at EL=80.

GCSR says wfsc000549 is saturated why? Steve says the seeing is too good, but I don't believe him.

DX M2 Globals are: Xglob=+1.350 Yglob=+1.900 Zglob=-0.200 RX=78.3 RY=264.9

GStar wfsc #
0 536-542
23 544-546
0 547-548
28 549-551
0 552-553
36 554-556
0 557-558
33 559-561
0 562-563
8 564-566
0 567-569
19 571-573
0 574-575
42 576-578
0 579-580
39 581-583
0 584-585
43 586-588
0 589-591
44 592-594
0 595-596
14 597-599
0 600-601
12 602-604
0 605-607
41 608-610
0 611-612
22 613-615
0 616-617
24 618-620
0 621-622
20 623-625
0 626-627

11:42 finished

 => Field Data Results
      Motion of the secondary need to remove bi-nodal astigmatims
              From Astigmatism Measurement
          Tip/R_x = -70.5", Delta IE =  70.5"
         Tilt/R_y = -82.6", Delta CA =  82.6"
Plane Fit: Ax + By + C = z
A =       2.58613
B =      0.260202
C =       183.406
s =       166.490
              From Focus Measurement
          Tip/R_x =  -3.9", Delta IE =   3.9"
         Tilt/R_y = -38.8", Delta CA =  38.8"

John doesn't understand these results as our adjustment doesn't seem to have changed much.

Getting the signs right

Empirical verification on sky

+100 RX requires -21 IE to correct
+100 RY requires +21 CA to correct


11:52 Rest shell and Close dome. T=-1.0 degC, Sky is clear.

The Plan

Here's the Night Guy's view of what is going to happen on Sunday during the

We'll leave the telescope parked at horizon with DX M2 deployed so
you can install the retroreflector on the right side.
Then you'll move it to zenith for:
        a) filling PISCES
        b) AO calibration
        c) removing plastic from the front of AGW1

At 4:00 PM we will move back to horizon for:
        a) removing the retroreflector [Rick, et al.]
           (needed since TCS needs to use LBCs for a couple hours)
        b) swapping UMAC CPU from DX M2 to SX M2 [John and Steve]
        c) filling LBCs with LN2 [Doug]
        d) switching over to BP12+ [Stephen]
           (BP11 needs to run until 4PM because the AO guys are using it.)

Then we'll move to zenith to reconfigure swing arms.
(Norm: do you want LBC or LDG first? I think LFBG is still non-functional.)

We'll plan to open the dome just before sunset at 6:05 PM.

-- JohnHill - 09 Oct 2011
Topic revision: r4 - 09 Oct 2011, JohnHill
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback