AGw Commissioning with IRTC 31 October - 01 November 2008 MST

Observers: J. Hill on mtn and A. Rakich in Tucson
Software Support: N. Cushing in Tucson plus P. Grenz and J. Borelli at beginning
Telescope Operator: D. Gonzalez Huerta


The updated versions of PMC seems to have supressed the mirror oscillation problem, although the potential of it is still there. Seeing was 0.6 arcsec most of the night with a patch of 1 arcsec seeing near midnight. We definitely see the 0.3 arcsec HBS jumps without local capillaries on any elevation pads.

Had problems with IIF devolving from its preset state at 01:08 and 12:38 UT. Otherwise telescope and software were beautifully behaved. GCS had an acquisition failure at 12:44 UT. Minor problems with PCS connecting polys to MCSPU at beginning.

We are puzzled why the elevation pointing zeropoint IE has been drifting monotonically for three days. Especially when these three days have not changed temperature significantly. We see a focus drift of 0.4 mm through the night. I think this is because the windbracing superstructure is warming during the day, and then cooling during the night. Beyond that gradient effect, the new collimation model is excellent in weather like this with good temperature equilibrium.

We have converged on the AGw probe focus value for IRTC, and the answer is 36. We can add the decimal point after analyzing tonight and last night's data.

We measured scale factors for Zernike terms Z12 - Z22 and implemented them in IDL. We tested these in a variety of conditions. They generally converge well, but need to be supressed when the pupil is highly distorted by low-order terms.

We tested Doug's new pupil positioning algorithm in IDL. It definitely works better than the previous version, but we still found a few stars at large radii where it didn't converge.


Times reported in UT below

23:22 Rebooted lbtmu107 to clean up swap space, etc.

23:46 netconfig start (on lbtmu107), then restarted GCSL and PSFR subsystems

00:10 Running version 4.1 of PMCL and PMCR with telemetry turned off.

00:15 Restarted ECS (for possible new libraries that were rolled back)


00:36 Open Dome. T=7degC D=-7degC, Wind light, sky mostly clear with broken clouds to SW

00:58 Reboot the IRTC Windows machine (to see if it cleans up the socket errors)

01:04 pointto M5_0855 gs=0 TelM="ACQUIRE" (no polys)

01:05 pointto M5_0855 gs=0 TelM="ACQUIRE" (no polys)

Oct 31 18:05:11 lbtmu05 irc[5199]: PCS set new target to RA: 5.940502998519889e+00 Dec: 1.890421381594705e-01 complete
Oct 31 18:05:11 lbtmu05 irc[5199]: MCSPU error returned - ERROR :  no polys are being received!

01:06 stop and restart PCS

01:07 pointto M5_0855 gs=0 TelM="ACQUIRE" - succeeded

01:08 GCSGUI acquire window disappeared after David enters IE/CA. We think this might be because mount went off-source and the preset state devolved to none.

01:11 pointto M5_0855 gs=0 TelM="ACQUIRE" - succeeded and now we see a star. Adjust IE=+58 CA=-34. IE continues to get bigger every day.

01:20 Go to reset the az/el encoders to see if the IE correction changes.

01:35 Restarted IRTC software and set temp to 240 K.

01:46 pointto M5_0855 gs=0 TelM="ACQUIRE" PAM- succeed Adjust IE=+53 CA=-34 Zglobal=+0.49
So the drift in IE does not seem to be encoder drift.

01:52 pointto BS9188 gs=14 ACTIVE (PA=230) x=0 y=-113.5 focus=25.657 (star too close to background)

01:56 setExpTime 4000

01:56 pointto BS9188 gs=14 ACTIVE (PA=230) x=0 y=-113.5 focus=25.657 (now only a warning about the star brightness)
Focus is 32 in AIP_L.cfg.

01:58 Can't quite hold at 240K in this summer weather, so relax IRTC temp to 245 K.

02:00 GCS cycle time is 64 sec, so the delay does not seem to be the swapping on lbtmu107, as presently 0 swap in use. We have GCSL, idle PSFR, one and several tails of syslog running. CPU is 65-80% idle.

top - 19:03:06 up  2:38,  6 users,  load average: 4.38, 3.38, 2.92
Tasks: 100 total,   1 running,  99 sleeping,   0 stopped,   0 zombie
Cpu(s):  4.1% us,  1.8% sy,  0.0% ni, 52.6% id, 41.4% wa,  0.1% hi,  0.0% si
Mem:   1034552k total,   287896k used,   746656k free,    40128k buffers
Swap:  2621432k total,       84k used,  2621348k free,    96288k cached

 3975 telescop  15   0  8100 2880 1384 S  4.3  0.3   4:02.02
 3748 telescop  15   0  406m  64m 8472 S  3.0  6.4   2:22.34 GCSL
 3736 telescop  15   0  8208 2700 2408 S  2.2  0.3   1:41.00 gshmserver
 3932 telescop  15   0 10452 2868 2208 S  0.8  0.3   0:04.15 sshd
 3979 telescop  15   0 22352  11m 5216 S  0.7  1.1   0:05.33 ds9
 2346 root      15   0  1656  764  632 S  0.5  0.1   0:01.63 syslogd
  241 root      15   0     0    0    0 S  0.3  0.0   0:01.99 kjournald
 3707 telescop  16   0  9212 2636 2136 S  0.2  0.3   0:01.21 sshd
 3737 telescop  15   0 28136 2676 2288 S  0.2  0.3   0:04.20 rpcserver
    1 root      16   0  1944  560  480 S  0.0  0.1   0:00.80 init

Seeing is 0.6 FWHM in 4.6 sec Guider exposures.

Conventional Focus Curves for IRTC - AGw Probe Focus

02:09 focus 0 (to offload active corrections from M2)

02:12 pauseGuiding

02:14 cl < (oops, those cubes went in some funny directory on IRTC machine.)

02:17 resumeGuiding

02:19 focus 0; pauseGuiding; stop Grab mode

02:20 cl < - The immediate visual conclusion of this sequence is that the focus is better near the beginning of the sequence at M2 dZ=+0.045 mm and gets worse as we move toward dZ=-0.045 mm.
irtc.20081101.022009.fits  +0.045 cube 0002
irtc.20081101.022114.fits  +0.030 cube 0003
irtc.20081101.022218.fits  +0.015 cube 0004
irtc.20081101.022316.fits  +0.000 cube 0005
irtc.20081101.022426.fits  -0.015  cube 0006
irtc.20081101.022540.fits  -0.030  cube 0007
irtc.20081101.022643.fits  -0.045  cube 0008
irtc.20081101.022735.fits  blank sky cube 0009
A problem in this script is that the offset commands resume guiding and therefore WFSing. So maybe I can only stopGuiding before running it?

02:35 Set focus back to 29 in AIP_L.cfg

02:36 pointto BS9188 gs=14 TelM="ACTIVE" focus=22.660 - succeeded. Seeing is stil 0.6 arcsec. Focus looks worse by eye after collimation at the nominal location. (29 on-axis)

02:45 focus 0

02:48 stopGuiding; cl < (first image had a socket error)
error   +0.045
irtc.20081101.024906.fits  +0.030 cube 0010
irtc.20081101.024957.fits  +0.015 cube 0011
irtc.20081101.025039.fits 0.000 cube 0012
irtc.20081101.025131.fits -0.015 cube 0013
irtc.20081101.025221.fits -0.030 cube 0014
irtc.20081101.025316.fits -0.045 cube 0015
irtc.20081101.025405.fits blank sky cube 0016

03:02 Set focus down to 26 in AIP_L.cfg

03:03 pointto BS9188 gs=14 TelM="ACTIVE" focus=19.663

03:08 focus 0; The net offload on M1 is X=+0.046 Y=-0.147 Z=-0.209 (rel to 0.490 Zglobal) RX=-1.58 RY=-0.49

03:10 stopGuiding; cl < (third or fourth offset failed)
This sequence has the same result as before. The first image is nearly in focus, and by the time we get to the last one, it is a donut several arcsec across (hardly visible in the 1 sec exposures).
See the answer at 03:36 UT below.
irtc.20081101.031106.fits +0.045 cube 0017
irtc.20081101.031200.fits +0.030 cube 0018
irtc.20081101.031249.fits +0.015 cube 0019
irtc.20081101.031332.fits +0.000 cube 0020
irtc.20081101.031425.fits -0.015 cube 0021
irtc.20081101.031520.fits -0.030 cube 0022
irtc.20081101.031614.fits -0.045 cube 0023
irtc.20081101.031705.fits blank sky cube 0024

03:20 pointto BS9188 gs=14 TelM="ACTIVE" focus=19.663 (to restart guiding and WFSing)
This position is clearly out of focus on the IRTC when the AGw is collimated.

03:28 takeseq exptime=1 Nimages=20 take1+ irtc.20081101.032821.fits cube 0025

03:30 Set focus up to 35 in AIP_L.cfg

03:20 pointto BS9188 gs=14 TelM="ACTIVE" focus=28.665
For sure this IRTC image is sharper than the previous one.

03:36 stopGuiding; cl < (fourth offset failed? was it the "+5"?)
Finally the world is making sense. Eyeballing these images as they are collect looks like the IRTC focus is either 35, or 1-2 mm higher. For sure this is the data to analyze to make a nice focus plot. A positive value for the secondary dZ offset means that the probe focus position has to move more positive (higher).
irtc.20081101.033741.fits +0.045 cube 0026
irtc.20081101.033833.fits +0.030 cube 0027
irtc.20081101.033924.fits +0.015 cube 002
irtc.20081101.034006.fits +0.000 cube 0029 (really +0.015 since the offset failed)
irtc.20081101.034058.fits -0.015 cube 0030
irtc.20081101.034151.fits -0.030 cube 0031
irtc.20081101.034244.fits -0.045 cube 0032
irtc.20081101.034333.fits blank sky cube 0033

Note added later by JMH: on 06-Nov-2008 Dave T. measured the images in this sequence with the following results:
# LM2Zmm FWHMpix # image
0.1164462  5.66         # cube0026
0.1013839  4.25         # cube0027
0.0863717  3.90         # cube0028
0.0863825  3.59         # cube0029
0.0563911  4.36         # cube0030
0.0413847  6.47         # cube0031
The parabolic fit gives a best focus at 0.0813 mm LMZ2, or an offset of +0.010 mm of M2 above the designated probe focus of 35. 166 nm of Z4 moves focus 1 mm of F/15 focal plane shift (sign unspecified), and +166 nm of Z4 moves M2 -0.0062 mm at the hexapod (actual sign). Thus, the adopted focus of the AGw probe on-axis for IRTC should be 36.6 mm in AIP_L.cfg.

Checking Focus with Intra-/Extrafocal Pupils

03:48 pointto M5_0881 gs=4 TelM="ACTIVE" x=0 y=-143.6 focus=24.866 (warning about guide exposure too short = 1.4 sec)

04:00 takeseq exptime=0.001 Nimages=3000 frmrate=100 take1+ irtc.20081101.035956.fits cube 0034
There was an active optics update at frame 1400 in this 3000-frame cube (Default mode).

04:15 focus 0

04:19 cl < (extrafocal images failed on a IRTC socket error).

04:24 resumeGuiding

04:26 pauseGuiding; cl <
irtc.20081101.042644.fits intrafocal cube 0037
irtc.20081101.042823.fits extrafocal cube 0038
irtc.20081101.043006.fits blank sky cube 0039
Intrafocal is just a shade smaller than Extrafocal. 116/119 pixels

04:33 resumeGuiding

04:36 Set focus up to 38 in AIP_L.cfg

04:37 pointto M5_0881 gs=4 TelM="ACTIVE" x=0 y=-143.6 focus=27.885

04:41 focus 0

wfsc 0294 - 0297 has an oscillation of Z6 +-200 nm. GS 4 is bright, and we were doing 15 sec WFS exposures.

04:50 pointto M5_0881 gs=1 TelM="ACTIVE" x=0 y=-116.1 focus=31.361 - succeeded

wfsc 0307 had a jump of Z6 = 171 nm.

04:56 pauseGuiding; cl < (one of the sequences had an error)
Pupils show very strong dome seeing at the lower/forward edge of the mirror.

05:01 resumeGuiding

05:03 pauseGuiding; cl <
irtc.20081101.050450.fits  intrafocal cube 0042
irtc.20081101.050633.fits  extrafocal cube 0043
irtc.20081101.050815.fits  blank sky cube 0044
Intrafocal slightly larger in diameter than extrafocal. 122/115

Thus, the conclusion from this quick analysis is that 36 mm is the correct focus value for IRTC. (1 percent difference in diameter produces 100 nm of Z4, and 166 nm of Z4 moves focus 1 mm of F/15 focal plane shift.)

05:10 Slew to zenith to check treehouse door/vents.

05:24 Set focus to 36 in AIP_L.cfg

05:25 pointto M5_0881 gs=1 TelM="ACTIVE" - succeeded

05:31 setWFSExpTime 60000 to try to average out more dome seeing. We had a few good corrections, but it really doesn't seem to have improved the oscillation of Z4 and Z6.

05:39 pauseGuiding; move to 0.7 mm intrafocal.

05:43 Open both left vent doors

05:46 resumeGuiding; Note that we haven't sent corrections in 7 minutes, so the primary hasn't updated in 7 minutes. The the first correction of +440 nm Z4 is taking out what the look-up table would have put in. Then, after the lookup table is applied, the second correction is -536 nm.

Opening the left vent doors hasn't improved the seeing. Now about 0.7 FWHM on the Guider.

05:54 takeseq exptime=0.0025 Nimages=3000 frmrate=100 take1+

06:01 things are looking quite stable at wfsc 0330. Andrew is thinking that the seeing may be worse with vent doors open, but the averaging of the WFS in 60 sec may be better. However there are still variations (200 nm Z4) in the focus position in wfsc 331-332.

06:02 setWFSExpTime 30000 to see how that works.

06:06 Restarting SX mirror ventilation.

Optical fact of the day: If this average wavefront (100 nm rms) were the diameter of the earth, the biggest bump would be 15 cm (rms). (Optical guy with too little sleep.)

06:15 Seems stable with 30 sec WFS exposures.

06:16 Close left vent doors

06:16 focus 0; The net offload on M1 is X=+0.067 Y=-0.168 Z=-0.377 (rel to 0.490 Zglobal) RX=-1.81 RY=-0.73

06:21 wavefront for 30 sec exposures is up above 200 nm rms again, especially the Z5/Z6 astigmatism terms.

06:26 image is significantly out of focus. Big bubble of bad seeing? Guider image was up to 1.2 arcsec at EL=56 deg. Polaris monitor did also show a bump of bad seeing, but it was reporting 3.5 arcsec.

Measuring higher order wavefront coefficients

Move to higher elevation in search of better seeing.

06:46 pointto BS9107 gs=46 ACTIVE

06:51 pointto BS9107 gs=0 ACTIVE

07:00 pointto BS9107 gs=0 GUIDE

Andrew running agw_collimate up to wfsc 0008

07:04 open all 4 vent doors

07:11 Z11=+500 wfsc 0009 - 0011

07:12 cancel the Z11 and recollimate wfsc 0012 - 0013

07:15 Z12=+500 wfsc 0014 - 0016 (measured 118 nm)

07:18 cancel Z12 and recollimate wfsc 0017 - 0022 (focus bouncing a bit)

07:24 Z13=+500 wfsc 0023 - 0025 (measured -140 nm)

07:27 cancel Z13 and recollimate wfsc 0026 - 0028

07:31 Z14=+500 wfsc 0029 - 0031 (measured 172 nm) wfsc 0032 is bonus measurement

07:35 cancel Z14 (some confusion about when this was sent) and recollimate wfsc 0033 - 0039

07:44 Z15=+500 wfsc 0040 - 0042 (measured -170)

07:46 cancel Z15 and recollimate wfsc 0043 - 0049

07:53 Z16=+500 wfsc 0050 - 0052 (5th degree coma, measured +57)

07:56 cancel Z16 and recollimate wfsc 0053 - 0055

07:59 Z17=+500 wfsc 0056 -0058 (measured

08:02 cancel Z17 and recollimate wfsc 0059 - 0064

08:07 Z18=+500 wfsc 0065 -0067 (measured +101)

08:10 cancel Z18 and recollimate 0068 - 0070

08:13 Z19=+500 wfsc 0071 -0073 (measured -98)

08:16 cancel Z19 and recollimate 0074 - 0076

08:19 Z20=+500 wfsc 0077 - 0079 (measured +160)

08:21 cancel Z20 and recollimate wfsc 0080 - 0084

08:26 Z21=+500 wfsc 0085 - 0087 (measured -149, -153, -161)

08:30 cancel Z21 and recollimate wfsc 0088 - 0091

08:34 Z22=+500 (bad forces - same as on 18-19 October)

08:35 Clear C00 and recollimate wfsc 0092 - 0095 (Z22baseline=+46)

08:38 Z22=-500 wfsc 0096 - 0098 (measured gross -118, -112, -100 with some coupling to Z11)

08:41 cancel Z22 and recollimate wfsc 0099 - (these corrections are dominated by oscillations in trefoil at the 40-60 nm rms level.)

Ship the scale factors to Doug for inclusion in IDL. Our initial plan is to divide these measured scale factors by 2 so that they have a damping gain of 0.40 when combined with our standard value of 0.80. These scale factors live in /home/LBTO/agw/idl/wfsc/

08:59 jump at wfsc 0116

09:00 pointto BS9107 gs=0 ACTIVE

Testing the new Zernike scale factors in IDL

09:12 pointto BS9107 gs=0 GUIDE

09:15 agw_collimate with IDL updated with the new scale factors and new pupil position algorithm. wfsc 0118 is the first image with corrections being applied through Z22. Something that looks like cross-coupling at the beginning, but seems to have settled down.

09:30 wfsc 0127 counting all aberrations from Z4 to Z22 had 61 nm rms wavefront.

09:35 wfsc 0131 counting all aberrations from Z4 to Z22 had 52 nm rms wavefront.

Testing new pupil equation off-axis

09:39 pointto M5_1133 gs=6 TelM="GUIDE" x=0 y=181 focus=19.979
first IDL image was wfsc 0134. Andrew thinks the pupil is in the wrong place by a row. This affected Z16/Z17 the most. Maybe we have some Z8 - Z16 crosstalk, so the off-axis convergence is still not so hot.

09:49 focus 0

09:53 restart IDL collimating with wfsc 0141. Error in irc_sendwavefront. Zernikes are getting to PSF, but they aren't being applied to mirror because of bad new influence forces.

09:57 clear C00, and start again.

10:03 pointto M5_1133 gs=8 TelM="GUIDE" (probe too far out)

10:05 pointto M5_1133 gs=5 TelM="GUIDE" (no star on acquisition image)

10:06 pointto M5_1133 gs=0 TelM="ACQUIRE" PAM- - Adjust IE=+65 CA=-24

10:08 pointto M5_1133 gs=5 TelM="GUIDE" x=0 y=-156.8 focus=23.965 PA=171.38 - succeeded

10:10 restart IDL collimating with wfsc 0150. First correction makes a big correction to the image. Pupil looks better aligned this time.

10:17 collimated through wfsc 0155 and nicely converged.

10:18 pointto M5_1133 gs=5 TelM="GUIDE" PA=130 x=103 y=-117.6 focus=23.964 - succeed Resume collimating with IDL.

10:23 collimated through wfsc 0160

10:24 pointto M5_0182 gs=10 TelM="GUIDE" PA=INDEF=192.64 x=0 y=-189.7 focus=18.413

10:28 agw_collimate_loop, EXP=30 s Pupil is at 8,7

10:40 stop collimating with wfsc 0169 - not very good convergence

10:41 pointto M5_0182 gs=9 TelM="GUIDE" PA=INDEF=130.65 x=0 y=-187.0 focus=18.907

10:42 agw_collimate_loop, EXP=30 s starting with wfsc 0170. Pupil still looks to be at 8,7 but convergence is better here.

10:57 stop collimating at wfsc 0183

10:59 pointto M5_0182 gs=8 TelM="GUIDE" PA=INDEF=298.1 x=-0.3 y=-162.7 focus=23.041

11:01 agw_collimate_loop, EXP=30 s starting with wfsc 0184. Pupil is well centered.

11:07 Have analyzed data from October 19th for low-order scale factors, and added these results to the high order data. Excel file attached. Interesting to note the Z11 term. For every 1000 nm of Z11 sent we get an average of 745 nm of Z11 measured, AND -1728 nm of Z4 as well.

11:12 stopped collimating on gs=8 at wfsc 0193. The convergence was OK, but the seeing got worse.

11:13 pointto M5_0182 gs=7 TelM="GUIDE" PA=INDEF=69.46 x=0 y=-146.6 focus=25.467

11:14 agw_collimate_loop, EXP=30 s starting with wfsc 0194. Pupil is centered.

11:21 stop collimating at wfsc 0200 with OK convergence.

11:23 pointto M5_0182 gs=6 TelM="GUIDE" PA=INDEF=327.94 x=0 y=-139.1 focus=26.501
Collimation from the previous star left this one looking OK.

11:23 agw_collimate_loop starting at wfsc 0201. Pupil is centered.

11:36 stop collimation at wfsc 0212

11:37 pointto M5_0269 gs=6 TelM="GUIDE" PA=INDEF=31.51 x=-0.6 y=-188.6 focus=18.633

11:40 agw_collimate_loop starting at wfsc 0213, pupil is at 8,7

11:51 stop collimating at 0218.

11:52 Clear C00 force corrections from primary

11:53 agw_collimate_loop starting at wfsc 0220. First two corrections made excellent improvement to the image.

11:59 focus 0 after wfsc 0225 but didn't seem to work. And doesn't appear in irc syslog, so the problem was that Doug's IDL routine was talking to irc, so the irc command sent by IRAF didn't get through. I think Jose told us that it would work like this with only one irs object per instrument.

12:05 focus 0 after wfsc 0230 leaves the numbers still in the M2 active optics vector.

12:07 stop collimating after wfsc 0231

12:08 focus 0 worked correctly

12:09 agw_collimate_loop starting at wfsc 0232.

12:21 stop collimating with not good, but not horrible convergence at wfsc 0244.

12:23 pointto M5_0269 gs=3 TelM="GUIDE" PA=INDEF=305.52 x=0 Y=-175.4 focus=20.96

12:25 agw_collimate_loop starting at wfsc 0246. With Andrew running post_collimate in parallel (is that legal?). Note that John and Andrew are getting two different numbers between agw_collimate loop and post_collimate. (Z6 different by factors of 3) Andrew stopped running post_collimate after wfsc 0254.

12:38 GCS drops out of guiding around wfsc 0256. It looks like IIF devolved out of guide state.

Comparing old pupil algorithm with new one

Work on the same off-axis star with GCS in ACTIVE mode rather than collimating with IDL as above.

12:44 pointto M5_0269 gs=3 TelM="ACTIVE" (lost star in the acquisition offset) acquire 0041

12:46 pointto M5_0269 gs=3 TelM="ACTIVE" (same preset worked just fine the second time) acquire 0042

GCS WFSing diverged into some horrible trefoil mess, while the new IDL algorithm had struggled but succeeded on the same star.

12:53 agw_collimate to try to recover the horrible image. Problems with sending large Z16, Z17 corrections when the pupil is highly distorted. We clearly need to cut off the high order corrections until the low order terms have converged.

12:56 clear C00 so the image only has large coma. This still gives a big Z16 correction. wfsc 0264+- Pupil is at 7,8. Z16 seems to be running away.

13:01 clear C00 to get rid of bogus Z16


13:04 stopGuiding (pressed button on GCSGUI)

13:10 Close Dome T=6.0degC D=-8degC Wind moderate, Sky photometric.

13:20 Lots of RPC errors spewing out of lbtmu107 when David shuts down the AGw. We stopped GCS for this reason.

-- JohnHill - 01 Nov 2008
Topic revision: r8 - 06 Nov 2008, JohnHill
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback