AO Commissioning on 15 June 2011

Observer(s): G. Brusa, E. Pinna, J. Brynnel, F. Quiros-Pacheo, J. Argomedo, K. Boutsia (LBTO)

Telescope Operator: D. Gonzales (LBTO)

SW Support: N. Cushing, D. Cox (Tucson)

Telescope Support: J. Little (LBTO)

Instrument Support: D. Officer (LBTO)

Summary

We had a good night from the point of weather and seeing (with values from DIMM of 0.8" at 60 degrees elevation). Despite several problems from both the telescope and the AO system we were able to do the following:

  • take some PISCES flats and standard stars in open loop
  • complete the slope offset measurement task
  • test and troubleshoot the new automatic gain routine for bright stars
  • take measurements for the optical gain in the bright stars regime

The telescope had two main problems: * temperature compensation of telescope structure had to be disabled due to the fact that some of the temperature sensors had been disconnected by the day crew. * we had three instances of rotator fail, this was fixed by Dan Cox.

The AO system' main issue as that the shell was 'ripped' many times, usually during the star acquisition task. In some instances we were able to figure out the cause, i.e. anomalous reading from temperature sensors, in other cases we were only able to determine that an over-current had occurred without being able to determine the cause of the over-current. -- GuidoBrusa - 15 Jun 2011

Details

23:42 darks with the dome lights ON and they are filling the dewer.

00:25 Piscses temperature reading 75.20K

01:19 Piscses temperature reading 75.00K
  • we are still taking darks with the lights ON
  • first set of darks is over pisces.*.0001.fits - .0145.fits but with the lights on and with varying temperature

02:24 Lights off in the dome - take some darks for comparison.
  • took some 60sec darks starting with *.146.fits - *.155.fits

02:43 Starting twilight flats - it is a bit late and the counts are low
  • repeat at dawn

03:00 - problems with the hexapod - T.O. cannot move them
  • restart the AOS - initialize hexapod
  • there is a big amount of temperature corrections ... (?)

03:16 homing and initializing hexapod again
  • temperatures are strange

03:19 still no progress - calling chris
  • it seems that the thermal computer is not working properly
  • we have to desable temperature correction in both mirrors
  • decided to go on without temperature corrections

03:45 iif_preset, S867, MODE=ACE, GS=10, RS=0, PA=-50
  • no guide star - check pointing
  • S867 (HST) - BS9155

03:57 iif_preset, S867, MODE=ACE, GS=5, RS=0, PA=-50
  • iif_offset, 5, -1, COORD='DETXY'
  • cannot offset the telescope
  • err = Resmsg[1] = AOARB:Illegal command for state ReadyToAcquire (Pause)
  • err = Resmsg[2] = Pause[23] failed - AOARB:Illegal command for state ReadyToAcquire (Pause)
  • we cannot send the offset from the telescope iraf either

04:07 iif_preset, S867, MODE=ACE, GS=5, RS=0, PA=-50
  • I could not offset because the loop was not closed - I should have done active preset!!!

04:10 iif_preset, S867, MODE=ACQUIRE, TS=0, GS=5, PA=-50
  • iif_offset, 5, -1, COORD='DETXY'
  • iif_offset, -5, -12, COORD='DETXY'
  • iif_offset, -7, -13, COORD='DETXY'

04:40 AGW diconnected - we have to start again and send the preset

04:41 iif_preset, S867, MODE=ACQUIRE, TS=0, GS=5, PA=-50
  • iif_offset, -7, -1, COORD='DETXY' - too much offset
  • iif_offset, -5, -3, COORD='DETXY'
  • iif_offset, -9, -6, COORD='DETXY' - we use this for the last set

04:55 iif_preset, S867, MODE=ACE, GS=5, RS=0, PA=-50
  • preset failed - rotator crushed - it has to be reset from the treehouse
===> Error in irc_presetTelescope
        err = Resmsg[0]  = PresetTelescope result status: Error
        err = Resmsg[1]  = Stop[26] complete
        err = Resmsg[2]  = PCS failed StartTrackingRotator on side right because ERROR rotator is not movable (Not 'READY' for some reason)..
        err = Resmsg[3]  = PCS StartTracking failed on side right with DEFAULT.

05:09 - rotator fixed... shell ripped
  • probably the shell ripped because the anemometer does not give a read
  • the anemometer went off at 5:05

summary standard
pisces.20110615.0171.hdr.fits - *.0176.fits J
pisces.20110615.0195.hdr.fits - *.0206.fits J
pisces.20110615.0229.hdr.fits - *.0232.fits J
pisces.20110615.0177.hdr.fits - *.0181.fits H
pisces.20110615.0191.hdr.fits - *.0194.fits H
pisces.20110615.0207.hdr.fits - *.0210.fits H
pisces.20110615.0225.hdr.fits - *.0228.fits H
pisces.20110615.0182.hdr.fits - *.0190.fits Ks
pisces.20110615.0211.hdr.fits - *.0224.fits Ks

05:13 iif_preset, S867, MODE=ACE, GS=5, RS=0, PA=-50

05:16 irc_runao
 
===> Error in irc_runao
        err = Resmsg[0]  = RunAO result status: Error
        err = Resmsg[1]  = RETRY: No stars found in TV frame (-20001) WFSARB_STAR_NOTFOUND
        err = Resmsg[2]  = AcquireRef[30] failed - RETRY: No stars found in TV frame (-20001) WFSARB_STAR_NOTFOUND
  • the AO team looked for the star but it was not at the offset same position as yesterday
  • we decide to go on with their program

start of AO tests
05:22 iif_preset, Pisces_14, MODE=ACQUIRE, GS=0, ROTMODE=IDLE
  • correcting pointing

05:31 iif_preset, Pisces_14, MODE=ACE, GS=5, RS=0, PA=-150

05:34 irc_runao - success!
  • DIMM 1.0''

05:38 - the preset is cancelled -
  • the loop was closed and then they received a stopao from the AOS
  • the loop opened but we don't know why
  • call Doug Miller

05:41 iif_preset, Pisces_14, MODE=ACE, GS=5, RS=0, PA=-150
  • preset failed - because the rotator is in a fault state - T.O is calling someone about this...
  • it looks like a software problem
===> Error in irc_presetTelescope
        err = Resmsg[0]  = PresetTelescope result status: Error
        err = Resmsg[1]  = Stop[37] complete
        err = Resmsg[2]  = PCS failed StartTrackingRotator on side right because ERROR rotator is not movable (Not 'READY' for some reason)..
        err = Resmsg[3]  = PCS StartTracking failed on side right with DEFAULT.
According to Doug Miller
  • the iif found out that the rotator stopped working or the telescope was not tracking and it sent a stopao that was the correct thing to do...
  • it seems to be a PCS-MCS problem

06:08 - David and AO team are going to the dome to fix the problem

06:21 They are back - they have swapped the fibres in the racket at the control housetree (??)

06:24 iif_preset, Pisces_14, MODE=ACE, GS=5, RS=0, PA=-160

06:26 irc_runao - success

06:47 iif_preset, Pisces_14, MODE=ACE, GS=5, RS=0, PA=-160

06:48 irc_runao - failed
  • err = Resmsg[0] = RunAO result status: Error
  • err = Resmsg[1] = err = Resmsg[2] = AcquireRef[47] failed

06:52 iif_preset, Pisces_14, MODE=ACE, GS=5, RS=0, PA=-160

06:53 irc_runao - failed giving the same error

06:58 if_preset, Pisces_14, MODE=ACE, GS=5, RS=0, PA=-160

06:59 irc_runao - success

07:15 DIMM 0.9''

07:24 DIMM 0.85''

07:34 iif_preset, Pisces_14, MODE=ACE, GS=5, RS=0, PA=-160

07:35 irc_runao - success

07:49 iif_preset, Pisces_14, MODE=ACE, GS=5, RS=0, PA=-160

07:51 irc_runao - success

08:08 iif_preset, Pisces_14, MODE=ACE, GS=5, RS=0, PA=-160

08:09 irc_runao - failed - it ripped on its own
===> Error in irc_runao
        err = Resmsg[0]  = RunAO result status: Error
        err = Resmsg[1]  =          err = Resmsg[2]  = AcquireRef[73] failed

08:12 iif_preset, Pisces_14, MODE=ACE, GS=4, RS=0, PA=30
  • changed position angle and GS because we were reaching the Pisces limit

08:16 irc_runao - failed giving the same error as before
  • the ripping is probably due to the BCU voltage - in the housekeeping

08:20 iif_preset, Pisces_14, MODE=ACE, GS=4, RS=0, PA=30

08:21 irc_runao - success

08:45 Pisces temperature 74.72K

08:48 iif_preset, Pisces_14, MODE=ACE, GS=4, RS=0, PA=30 - failed
  • hexapod had to be reinitialized
 
===> Error in irc_presetTelescope
        err = Resmsg[0]  = PresetTelescope result status: Error
        err = Resmsg[1]  = Stop[82] complete
        err = Resmsg[2]  = dx.adsc: [moveAbs] UMAC is not ready - brake may be set

08:52 iif_preset, Pisces_14, MODE=ACE, GS=4, RS=0, PA=30

08:55 irc_runao - failed
  • err = Resmsg[0] = RunAO result status: Error
  • err = Resmsg[1] = err = Resmsg[2] = AcquireRef[86] failed

08:56 iif_preset, Pisces_14, MODE=ACE, GS=4, RS=0, PA=30
  • preset failed because the shell was not ready

09:00 iif_preset, Pisces_14, MODE=ACE, GS=4, RS=0, PA=30

09:03 irc_runao - success

09:16 - check images .320.fits - .329.fits -
  • one in there shows zero background and a stripe with correct counts in the lower part
  • when this happens the ds9 the AO team is using, crushes
  • dimm 0.9''

09:55 if_preset, Pisces_14, MODE=ACE, GS=4, RS=0, PA=30
  • err = Resmsg[2] = dx.adsc: [moveAbs] UMAC is not ready - brake may be set
  • hexapod had to be reinitialized
  • this time the hexapof might have been switched off because the AO team turned off the power on the secondary mirror
  • this should NOT happen!

09:58 iif_preset, Pisces_14, MODE=ACE, GS=4, RS=0, PA=30

10:01 irc_runao - failed
  • problems with the temperatures!

10:10 iif_preset, AO674, MODE=ACQUIRE, GS=0, ROTMODE=IDLE

10:12 iif_preset, AO674, MODE=ACE, GS=4,RS=0,PA=90
  • guide star is out of reach
  • err = Resmsg[2] = AGw command status: AGW2: Requested position is Out of Range
  • err = Resmsg[3] = error: setProbePosition: couldn't move probe to position (SFP) (x,y,z): -35.6559, -73.4229, 28.2888 mm.
  • err = Resmsg[4] = guidestar list successfully changed but probe couldn't be moved to position.

10:14 iif_preset, AO674, MODE=ACE, GS=4,RS=0,PA=90 - failed for the same reason

10:15 iif_preset, AO674, MODE=ACE, GS=7, RS=0,PA=90 - failed
  • we changed guide star but now the shell ripped

10:17 iif_preset, AO674, MODE=ACE, GS=7, RS=0,PA=90

10:20 irc_runao - successful

10:39 - see image *.442.fits and *.443.fits - strange!
  • the *.444.fits - seem fine! - we need to find out why this is happening

10:48 hexapod initialize

10:52 iif_preset, AO674, MODE=ACE, GS=7, RS=0,PA=90

10:55 irc_runao - success!

HR8799
11:16 iif_preset, Wref11new, MODE=ACQUIRE, GS=0, ROTMODE=IDLE

11:20 iif_preset, Wref11new, MODE=ACE, GS=6, RS=0, PA=-90

11:21 irc_runao - failed
  • BCU voltage 0

11:24 iif_preset, Wref11new, MODE=ACE, GS=6, RS=0, PA=-90

11:27 irc_runao - success
  • we stop the rotator - but this ripped the shell
  • the loop closed without needing a preset

11:48 the loop is open

11:51 iif_offset,20, 0, COORD='DETXY'
  • we tried to offset to take a dark but it did not work
  • probably because the loop is open (?) and after an ace preset it expects the loop closed ...

11:53 we try to offset from the telescope iraf and it still cannot move...

11:54 iif_preset, Wref11new, MODE=ACTIVE, GS=6, RS=0, PA=-90 - failed
  • is it because I put RS while I was not in ACE??
  • err = Resmsg[0] = PresetTelescope result status: Error
  • err = Resmsg[1] = Stop[131] complete
  • err = Resmsg[2] = full CCD readout is fully saturated. Processing impossible.
  • iif_offset, 20, 0, COORD='DETXY'
  • err = Resmsg[1] = right Preset inactive

11:55 iif_preset, Wref11new, MODE=ACQUIRE, GS=6, PA=-90
  • iif_offset, 20, 0, COORD='DETXY'
  • this worked but the sky background had changed in the meantime
  • the images obtained now, could be used as H flats (5000counts)

12:05 flats in H, J, K

12:13 Dome closes - dark frames

summary flats and darks
evening flats - low counts filter
pisces.20110615.0156.hdr.fits - *.0160.fits Ks
pisces.20110615.0161.hdr.fits - *.0165.fits H
pisces.20110615.0166.hdr.fits - *.0170.fits J

dawn flats - good counts filter
pisces.20110615.0487.hdr.fits - *.0516.hdr.fits H
pisces.20110615.0517.hdr.fits - *.0527.hdr.fits J
pisces.20110615.0528.hdr.fits - *.0533.hdr.fits Ks

darks - LIGHTS OFF exptime
pisces.20110615.0535.hdr.fits - *.0544.fits 1 sec
pisces.20110615.0545.hdr.fits - *.0554.fits 2 sec
pisces.20110615.0555.hdr.fits - *.0564.fits 3 sec
pisces.20110615.0565.hdr.fits - *.0574.fits 4 sec
pisces.20110615.0575.hdr.fits - *.0584.fits 5 sec
pisces.20110615.0585.hdr.fits - *.0589.fits 7 sec
pisces.20110615.0590.hdr.fits - *.0594.fits 10 sec
pisces.20110615.0595.hdr.fits - *.0599.fits 15 sec
pisces.20110615.0600.hdr.fits - *.0604.fits 20 sec
pisces.20110615.0605.hdr.fits - *.0609.fits 30 sec
pisces.20110615.0610.hdr.fits - *.0614.fits 40 sec
pisces.20110615.0615.hdr.fits - *.0619.fits 50 sec
pisces.20110615.0620.hdr.fits - *.0624.fits 60 sec
pisces.20110615.0625.hdr.fits - *.0629.fits 100 sec

darks -LIGHTS ON exptime
pisces.20110615 sec5.hdr.fits - *.0015.fits 1 sec
pisces.20110615.0016.hdr.fits - *.0025.fits 2 sec
pisces.20110615.0026.hdr.fits - *.0035.fits 3 sec
pisces.20110615.0036.hdr.fits - *.0045.fits 4 sec
pisces.20110615.0046.hdr.fits - *.0055.fits 5 sec
pisces.20110615.0056.hdr.fits - *.0065.fits 7 sec
pisces.20110615.0066.hdr.fits - *.0075.fits 10 sec
pisces.20110615.0076.hdr.fits - *.0085.fits 15 sec
pisces.20110615.0086.hdr.fits - *.0095.fits 20 sec
pisces.20110615.0096.hdr.fits - *.0105.fits 25 sec
pisces.20110615.0106.hdr.fits - *.0115.fits 30 sec
pisces.20110615.0116.hdr.fits - *.0125.fits 40 sec
pisces.20110615.0126.hdr.fits - *.0135.fits 50 sec
pisces.20110615.0136.hdr.fits - *.0155.fits 60 sec

-- KonstantinaBoutsia - 14 Jun 2011

UT 3:30 problems with the hexapod. Apparently some of the temperature sensors were removed from the top of the C-ring and this causes the problem. What happens is that the hexapod uses the temperature compensation of the C-ring extensions. We decided to disable the temperature compensation (3:44).

UT 4:45 (~) the rotator of the RFBG got stuck and needed manual reset

UT 5:05 the anemometer stopped working (from log) 'frame parse error: missing parameters…'

UT 5:12 we discovered that the shell had 'ripped' as a consequence of the anemometer going down

UT 5:14 back with shell up and anemometer working

UT 5:14 preset on S867 IR photometric standard ESO

UT 5:15 collimating active optics

UT 5:16 active optics residual error below 600nm

UT 5:16 no star found on TV

UT 5:18 we move the WFS stages to find the star, we cannot find the star by simply moving the stages to the offset that was found yesterday (of about 8-10mm at the f/15). The most likely scenario is that the active optics finds a guide star different from the one specified by the preset because of some small errors between the pointing and the active optics probe positioning (and/or acquisition routine), this is also combined with the fact that we move the rotator to keep PISCES always 'up' .

UT 5:22 moving to another star, PISCES14.

UT 5:31 the active preset succeeded

UT 5:33 the active optics res. error is about 3500nm, we send the acquire

UT 5:34 star acquired on TV

UT 5:35 runao success

UT 5:36 stopao? we don't know why

UT 5:39 runao: result status error preset inactive

UT 5:40 sending a preset again: 'rotator not movable'

UT 5:43 rotator is in fault state… calling Tucson

UT 6:20 swapped fibers in the tree house (to fix the rotator problem)

UT 6:24 sending a preset for PISCES14

UT 6:26 active optics converged to 1300nm

UT 6:27 collimated, runao

UT 6:28 returned success

UT 6:31 auto-gain 1.6, 0.9, 0.9

UT 6:31 the star is too close to the left edge of the detector on PISCES

UT 6:33 offsetting using AOS by 6mm

UT 6:35 putting FeII filter, ~12000 counts on the secondary star with 1 s integration. pisces.20110615.0237.fits.

UT 6:37 trying auto-gain again, it failed and found too high gain for all three mode ranges

UT 6:49 shell 'ripped' after another preset, why?

UT 6:55 shell 'ripped' after another preset, why?

UT 7:02 preset successful

UT 7:03 auto-gain successful. The auto-gain routine had some problems in the way it was written we decided to make some changes and try again.

UT 7:04 deltax = 0mm deltay = -6mm. these appear to be the required offsets to have the star in the center of PISCES field when it is in the 'sweet spot' of TV

UT 7:10 10 frames taken

UT 7:11 dark acquired

UT 7:19 starting acquiring the optical gain for the five chosen modes

UT 7:24 seeing from DIMM 0.85" (at 70 deg elevation) telescope 79 deg.

UT 7:30 TSS activated (or is it just a wrong state displayed)?

UT 7:35 the housekeeping process was stopped and restarted

UT 7:35 preset again

UT 7:35 runao

UT 7:45 during the auto-gain the shell 'ripped', it appears that the pupil was not properly centered, is the pupil centering loop working properly?

UT 7:51 preset

UT 8:00 NCPA measurement

UT 8:27 several 'shell rip', after looking at the logs we find that the problem was BCUVoltage, found at 0. We found that the the housekeeper caused the 'shell rip' five times due to the BCUVoltage to 0 (it turns out, after discussion with Marco and Armando that this is just a consequence of the ASM having had an over-current).

UT 8:40 we found out that the WaterInlet temperature is reported wrong (33C or 77C), the other temps seem to be correct. We decide to change the threshold on the water inlet to continue working.

UT 8:50 the shell can now set.

UT 8:51 hexapod problem , it is re-initialized

UT 8:54 the active optics takes some time to converge, 2500nm rms

UT 8:55 we decide to runao with a residual around 2000nm.

UT 8:56 the ao loop did not converge…

UT 8:56 sending a new preset.

UT 9:14 starting again the NCPA, numbers are very small, the biggest is 9nm.

UT 9:50 shell 'ripped', the hub temp was read to be 77C…

UT 10:25 we moved to a new star AO674, it is at 70 deg elevation vs. the previous that at the end was around 50. Now all the temperatures seem to be OK. Why did the elevation change fix the problem? Either a kinked hose or some 'loose' board...

UT 10:42 while taking data with the optical gain the shell ripped but ao is still running. To stop ao we turned ASM off and on again, there should be a better way of doing this...

UT 11:15 completed the optical gain measurement for five modes.

UT 11:30 taken some (incomplete) planets data

UT 12:05 taking some flats in J, H, Ks

-- GuidoBrusa - 15 Jun 2011
Topic revision: r8 - 22 Jun 2011, KonstantinaBoutsia
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback