You are here: Foswiki>Operators Web>WebTopicCreator>CheckList (16 Sep 2020, SteveAllanson)Edit Attach

Operator Checklist for the start/end of the night at LBT

The following procedure should be performed each evening before observing. It should be carried out with the intent to have the dome open 45 minutes before sunset when the Lucifer observers need sky flats and for LBTI observing, or 10 minutes after sunset for LBC and MODS observing.

1. Check the almanac and ask the Observers about their daily requirements, i.e. darks, bias, sky flats. Update the whiteboard behind the TO station.
The nightly almanac can be found on the OSA Night Report Tool, or generated on obs1 as follows: in a terminal window,
  • cd /home/telescope/sallanson/almanac
  • ./almanac
Sanity-check the time, date, and location.

2. Check with the day crew about recent changes or other special conditions that might be present.

3. In ECS GUI set the Snow Melting (SM) system to Ventilation mode to remove daytime heat from the roof and shutters. Typically this would be 2 hours before opening. Also open the Level 4 Dampers (Stealth Exhaust Dampers)
Set the temperature setpoint to match anticipated outside ambient temp, located at ECSGUI > Snow Melting (SM) page. ( * -10C is a good catch-all since there's no active cooling. Maybe change it to something much cooler during the winter. * ) For the louvers go to ECS > DP (Dampers). Press [Open All].

4. Verify that the audible alarm in the control room has not been silenced during the day. And verify that Alarm Handler alarms are not silenced.
First, in ECS GUI > OPS, make sure [Audible Alarm] is OFF. Then make sure Audible Alarm switch is turned to "Active". (Above Emergency Shut-Off button behind obs1 in the ctrl. rm.). For the Alarm Handler GUI, verifying that the lower right radio buttons are deselected, and "Silence Forever" in the Setup menu is deselected will most likely make sure alarms are audible.

You can test the sound in the LSS GUI. In the "Audio" section, hit "Test".

5. Check the primary glass temps, and begin mirror ventilation when necessary.
You should have a sense of what the outside temperature will be when it comes time to open. See the Weather Guidelines for Opening Enclosure for details on maximum temperature difference. You may need to manually cool/heat the primaries with mirror ventilation prior to opening. To do so, do the following:
  • Start Mirror Ventilation in the ECS > MV page. ([Run] button for both Left & Right Pressure Blowers).
  • In the PMC Ventilation subGUI (PMC GUI > Detailed Control/Status section > [Ventilation] button if you haven't already opened it), manually change the Target value to your predicted sunset temp.
  • Be sure that the duct temperature (PMC Ventilation subGUI) is no more than 7C above/below the current glass temperature (PMC Ventilation subGUI > Glass Therms) (PID mode limits this.)
  • In the Mirror Ventilation - Setpoints and Status Panel window (ECS > MV > [Display Settings Panel] button to bring up window), check that the Temperature Deadband value is 5C to avoid getting "supply temperature out of range" warnings in the ECS log buffer. You can set the Deadpoint value to 1C when the glass temp. is close to outside ambient temp., but you run the risk of alarms.

You may encounter a situation where the outside temperature is drastically different than glass temperature. See the Weather Guidelines for Opening Enclosure for restrictions on temperature differences.

You should ALWAYS run mirror ventilation at the start of the night if the weather permits. This is true even for Adaptive Optics nights when the observers do not want mirror ventilation running during the science observations. On those nights, start mirror ventilation as soon as you can, and continue until a few seconds before they start collecting the actual science data.

For LBC, it is preferred to run mirror ventilation for at least half an hour before observations begin. This typically means starting mirror ventilation shortly after sunset - sometimes before opening the dome (ref Issue 4361). Starting mirror ventilation with the dome closed may not be helpful if the dome is warm compared to both the mirrors and outside ambient.

Now that the mirror ventilation intake ducts are inside the chamber, it is possible to run mirror ventilation (with the dome closed) even when there is fog/rain/snow outside. This can be helpful in cooling off the mirrors when a front is passing through. Take care that the mirror ventilation setpoint (Mirror Ventilation - Setpoints and Status Panel window, ECS > MV > [Display Settings Panel] button) is still at least 2 degC above the chamber dewpoint (ECS > OPS > Chamber section, or ECS > Views menu > Analog Monitoring Page, or ENV GUI > Temp Sensors tab > Chamber Air section). (PID mode limits this).

6. Do a walkthrough of the telescope following the checklist ( 004s004) located in the aux control room on level 5. Be sure that all lights are off on 4, 3U, 3L, 1, and in the aux bay.

7. Check the TCS Operating Procedures page on the Wiki (Operators). Make sure all the subsystems are running.

8. In case of a major power outage where computers go down, go to the TCS Full Shutdown and Startup Procedure to bring all the software subsystems back up. Contact the on-call software support person for help if necessary.

9. When switching from one instrument to another, check the necessary configuration of the swing arms.
OSS > Swing Arms section > [left] & [right] buttons to bring up left & right OSSGUI Swing Arms windows.

For example, to work with LBCblue, prime focus must be deployed while secondary and tertiary must be retracted. When using Lucifer, tertiary and secondary must be deployed and prime focus must be retracted. Follow instructions for Moving Swing Arms for how to do this. A visual check of the swing arms should be performed prior to moving them, e.g. check if retroreflector is in place, etc.
  • For LUCI,LBTI,LN,PEPSIPFU (@FBG, PEPSIPFU@RBG): PF=Retracted M3=Deployed M2 =Deployed
  • For LBC (@PF): PF=Deployed M3=Retracted M2 =Retracted
  • For MODS,PEPSIPOL (@DG): PF=Retracted M3=Retracted M2 =Deployed

10. Verify that OSS is talking to the M2 hexapods and M3 units. (You'll see green statuses for the M2s and M3s on OSSGUI). Initialize and home M2 hexapods and M3 unit as required.
If you don't see green statuses for the M2 & M3 mirror icons on OSSGUI: OSSGUI > Press M2 and/or M3 icons to bring up OSSGUI Secondary and/or OSSGUI Tertiary Mirror window > Press Init.

To home:

M2 hexapods
OSSGUI > M2 icon. In OSSGUI Secondary Mirror window, press "Init". This will initialize and home the M2 hexapods.

M3 unit
OSSGUI > M3 icon. In OSSGUI Tertiary Mirror window, press "Init". Once complete, press "Home Sel". Once complete, press "Home Mirror".

11. Authorize the requested instrument(s) on IIFGUI (LUCI, MODS, LBC, LBTI or IRTC).
Verify telescope is stationary, and at zenith.

IIF GUI > [Control] button to bring up IIF Control window.

12. Verify status of primary mirrors.
Check Status Summary section of PMCGUI. Some possible scenarios:

  • Panicked mirror, mirror is down (Weight = 0%).
    • Click Recover Panic. Cell status should change to 'Running'.
    • Click All Air On in the Air Control section of PMCGUI. Monitor the Air Pressure sub-GUI until all loops are On, and all values are between 100 & 120.
    • In the Actuators sub-GUI, click Enable All in the event of disabled actuators. Some may not enable until the mirror has been raised. These can be seen in the Disabled section of the PMC Actuators subGUI.
    • Click Auto-Raise in the Raise/Lower/Bump section of PMCGUI. Monitor the Weight section until the weight shows 100%.
    • If necessary, return to the Actuator sub-GUI to enable any remaining actuators.
      • If just one is still disabled, it can be enabled individually by entering the number in the Actuator field, and hitting Enable. Otherwise, Enable All can be pressed again.

  • Disabled actuator(s).
    • Mirror status is yellow. Click Enable All in the Actuators sub-GUI.

  • Some other scenario.
    • Hover mouse over status to get a description of yellow/red warnings.
    • React accordingly.

13. Clear all active optics corrections from the previous night on the PSF Primary Mirror Active Optics and Collimation Control subGUI for both telescope sides.
  • Clear Active Optics on Primary
  • Remove all corrections from Pointing Corrections, Instrument Offsets, Secondary Offload, etc. on Primary
  • Verify the lookup table and temperature corrections are enabled.
  • Click 'initialize' for the Lookup Tables under both Position Details AND Figure Details in order to restore global offsets (you may want to note the previous nights values in case of similar conditions). Reset Inputs, Collimate.

14. Zero the relative encoders on the primary mirror hardpoints.
  • do this ONLY when the mirror is up in normal operating condition.
  • click Zero RE in the Hardpoints sub-GUI of PMCGUI.
  • You may need to periodically do this throughout the night after large changes in elevation. This is done to prevent the endless "primary mirror collimation error"s that arise from the encoders being slightly offset (you will usually see the total collimation of RX or RY stay yellow in this case)

15. For Gregorian observing, Clear active optics corrections on the PSF Secondary Mirror Optics and Collimation Control subGUI.
  • Clear Active Optics on Secondary
  • Remove all corrections on Secondary
  • Verify the lookup table and temperature corrections are enabled.
  • Click 'initialize' in order to restore global offsets (you may want to note the previous nights values in case of similar conditions). Reset Inputs, Collimate.

16. For Bent Gregorian observing, Click Initialize on the PSF Tertiary Mirror Optics and Collimation Control subGUI.

17. For Gregorian observing, Set the 'Secondary Control' parameters in the left and right PSF arbitrators (PSF GUI > Control). Note that during normal operation, the default settings are usually correct.
For Both Sides:
  • Set the Secondary Control to 'Split' mode.
  • Set the Z11/Z22 offloads to 'M1'.
  • Set all other options to 'No'.
  • On a Commissioning night, double check the necessary requirements (specifically AO nights).

18. Restore last known pointing model IE/CA values.

  • Check your email for the last time we used your intended instrument with the current pointing model.
  • Restore IE/CA using the Pointing Tool

19. Startup the appropriate rotator(s).
  • in MCSGUI > Rotator > (focal station) > ON
  • in MCSPU: rcready lfbg (for example)

If the rotator is in a fault mode, check the rot f (front) or rot d (direct) or rot r (rear) pages in MCSPU for fault messages.
  • Check the MCR state, reset in URTH if necessary.
  • You can reset various elements in MCSGUI > Rotator > (focal station) > Modal Commands subGUI.
  • A restart of MCSPU may be necessary (loop counts not updating and flashing red is a good indication).

20. Start the appropriate AGW(s) and home the stages.
ssh to the AGw server by typing ssh oac@oac (TO will have password). See Commissioning.AGwStages for additional details.

The following instructions show X as the AGw unit in question: 2=RFBG; 1=LFBG; (4=LDG, but MODS takes care of it's own AGw control)
  • The AGw units are usually left running, but if needed:
    • startAGW -u X (wait a full minute for things to power on).
    • To check if the AGw is running, a simple command like getxy -u X will let you know if communication is established.
  • home -u X -m 21

You may encounter some kind of communication error, which in most cases can be resolved by restarting the OACserver. See Commissioning.AGwStages for instructions. In all likelihood, the associated GCS subsystem will need to be restarted after this.

For MODS:
Because the AGw units for MODS are embedded inside the instruments, AGw startup requires using the MODS interface:
  • Make sure that the MODS instrument mechanism service (mmcService) and AGw stage server (agwServer) are running (ask the support astronomer).
  • Make sure that the MODS Control Panel (modsUI) is running (ask the support astronomer or observer).
  • Ask the observer or support astronomer to put MODS in OBSERVING mode. This will retract the internal calibration unit and open the instrument hatch to the sky, and unlock the AGw camera XY stage.
  • Restart the GCS, which will re-initialize the GCS/MODS interface and reset the MODS AGw XY stage (NOTE: MODS must be the authorized instrument on that side for this to occur).

These steps need to be completed before a test preset can be executed with MODS.

21. Startup the HBS, enclosure drive and telescope drives.
In MCSGUI:
  • Verify stowpins are disengaged in MCSGUI
  • HBS subGUI > Start HBS (may need to reset alarm)
  • in the System subGUI, monitor until the main pressure reads > 100
  • click enclosure drive on
  • in MCSPU monitor the er page until all bogie drives show ready/ON
  • click ready on AZ and EL drives (MCSGUI)
  • in MCSPU monitor the pos page until AZ and EL drives show ready

In MCSPU (monitoring values as above):
  • hbsstart
  • errot on
  • elready
  • azready

22. If not yet completed, fill LBC cameras and MODS1 with LN2.
In all likelyhood, this will end up happening prior to your walkthrough. Talk to the Instrument Support person on duty for the status of the fill.
  • Orient the telescope as necessary. Wait all drives and stop HBS in preparation for lockout.
  • Once completed, redo previous step to enable telescope motion.

23. Set the telescope-enclosure relative rotation setpoint to zero.

In MCSPU, monitoring the pos page:
  • ersetpoint 0
Verify the PVT value settles on 0.0.

24. Reset mount encoder zeropoints. This is done only at AZ=120.0 EL=90.0 degrees.
Follow the Resetting Encoders instructions on the Operators Wiki as time permits (could be before or after opening). In MCSPU, monitoring on the enc page:
  • azloadfromabs
  • elloadfromabs

25. Set telescope encoders to strip mode.
In MCSPU, monitoring the enc page:
  • elencsettype strip
  • azencsettype strip

26. Set the AdSec shells

  • Ensure AOS is communicating with the adsec as expected.
  • Ensure the AdSec is Ready and Safe to operate (AdSec Control GUI)
  • Click 'Set' on AOSGUI or 'setFlatAO' on the AdSec Control GUI.
  • Note the number of modes used to apply the flat in your log. Re-load the flat if necessary (AdSec Control GUI > Shape Control)

27. Send a test preset, or have the observers send a test preset to see that everything is working.
This could be done before or after opening. Follow the "Sending a preset in IRAF" instructions.

NOTE for MODS
If MODS is in Calibration Mode (hatch closed, calibration unit deployed, AGw stage locked), the test preset will fail, sometimes in confusing ways.

Ask the support astronomer or observer to put MODS into OBSERVING mode (open hatch, retract calibration unit, unlock the AGw stage), and then restart the GCS and wait for the reset to complete before issuing the test preset.

NOTE for LUCIFER
After sending a test preset, home the AGw stages so that calibrations can continue without being impeded by the probe on-axis.

28. Park the telescope at an AZ opposite sunset until you're ready to open.

See [https://www.gaisma.com/en/sunpath/tucson-arizona.png]

29. Move telescope to EL=30 pointing before opening shutter doors.

30. Weather permitting, press the permission button (Level 2 or Level 5), open the shutter and ventilation doors with ECS.
  • Put the wind at your back to open if the winds are high.
  • Click 'Open Both Doors' and confirm your choice in ECSGUI > SD.
  • Monitor latch stations for errors.

Check the Weather Guidelines for Opening Enclosure on the Operators Wiki page for further details.

31. Turn off all the observing chamber lights and heat traces with ECS.

32. Move telescope to the desired position for sky flats. (This is normally done with a preset from the instrument.)

33. Before the end of twilight, make sure all the dark shades in the control room and other parts of the fixed enclosure are closed.

End of the night

1. Slew telescope to EL=30deg (before closing shutter doors).
In MCSGUI:
  • Hold the AZ drive
  • type 30 in EL and click slew

In MCSPU:
  • elslewtohold 30

2. Close the Shutter Doors.
  • Put the wind at your back if the winds are high.
  • In the Shutter Door (SD) subGUI of ECSGUI, click Close Both Doors. Monitor the left and right latch station panels, to verify the process completes properly.

Check the Weather Guidelines for Opening Enclosure for further details.

3. Close Vent Doors if they were opened. (ECSGUI)

4. Stow the telescope and park the enclosure.
Park azimuth at 120.6 deg, and elevation at 90deg.

In MCSGUI:
  • type 90 in EL and click slew
  • type 120.6 in AZ and click slew

In MCSPU:
  • elslewtohold 90
  • azslewtohold 120

5. Turn off drives Azimuth, Elevation, and Enclosure drives.
In MCSGUI:
  • click Waitopr for Azimuth and Elevation.
  • click Drive off for Enclosure.

In MCSPU:
  • azwaitopr
  • elwaitopr
  • errot off

6. Turn off Mirror Ventilation. (ECSGUI)

7. Close Level 4 Dampers (aka "Stealth Exhaust Dampers" in TCS Control software)
ECS > DP > Close All

These are the 4 large tubes seen from outside. These were originally intended to have fans installed, but it never came to be.

8. Rest the shells on the Adaptive Secondaries (after dome shutters are closed and telescope is parked). Leave AdSec software running.

9. Reconfigure the telescope to ensure M2s are deployed (for AdSec safety) and LBCs are retracted (for filling)

10. Park and turn off the rotator(s).
Park Positions:
  • LUCI1 (LFBG) is parked at 341 degrees.
  • LUCI2 (RFBG) is parked at 210 degrees.
  • MODS1 (LDG) is parked at 210 degrees.
  • MODS2 (RDG) is parked at 210 degrees.
  • PEPSIPOL (LDG) is parked at 222 degrees.
  • PEPSIPOL (RDG) is parked at 222 degrees.

In MCSGUI > Rotator > (focal station) subGUI:
  • type park position angle (341 for example), click SlewToHold
  • click OFF

In MCSPU:
  • rslewtohold lfbg 341 (for example)
  • rcwaitopr lfbg (for example)

11. Stop HBS.
In MCSGUI > HBS subGUI:
  • click Stop HBS

In MCSPU:
  • hbsstop

12. Turn off the DIMM if you have been using it. (Now done automatically)

13. Turn on shutter door and drain pipe heat traces if temperature is below -3 degC, or there is a chance of snow. (ECSGUI > OPS)

14. Press the "End of Observing" button on the IIFGUI to put the PCS to sleep.

15. Stop pointing log if you started it.

16. Home AGw stages (to bring them all to the park position).

17. AGw units, M2 hexapods, M3 units and Primary Mirror cells are normally left running during the daytime unless an extended shutdown is planned.

18. Observers should start calibrations (if desired) at this point.

19. Turn on the lights in the chamber if no cals required.

20. Submit night report.

21. Silence the Alarm Handler (ALH) for the Day Crew.
Alarm Handler GUI > Setup menu > Silence Forever. ALH should be in this state only for the day, and it should be returned to its normal state upon the OSAs return at the beginning of the night.

-- JohnHill - 23 Nov 2011
-- SteveAllanson - 02 May 2012
-- SteveAllanson - 07 Feb 2014

-- %USERSIG{SteveAllanson - 2018-06-26}%
Edit  | Attach | Print version | History: r58 < r57 < r56 < r55 | Backlinks | View wiki text | Edit wiki text | More topic actions
Topic revision: r58 - 16 Sep 2020, SteveAllanson
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback