COVID-19 Shutdown Planning

Notes 2020-03-20

• Recap
  • Yang to plan swap tcs-test for Monday
  • Stephen WIP on http://info
  • Matthieu+IT obs1/cloud or not - choice for temp monitoring (also ask Peter about the power supply)
  • Carl to confirm with cc. IT for ARGOS
  • MODS/LUCI on Monday
  • OVMS few hours after everyhting is off

• status
  • tcs-test readiness to run ENV telemetry
    • IT to create the new telemetry mount and send the info to Yang
  • http://info.tucson readiness to pull webcams images
    • wda.sh and webcam, allsky moved to Tucson

• update http://info with a shutdown-centric view
  • access to all the webcams (inc. recent DX AdSec one in the clean room)
  • CR temperatures monitoring (moutain, Tucson)
  • link to dms telemetry
  • any other ideas

• Peter G. monitoring sensors
  • power supply at the mountain for new SensorPush temperature sensor system - MB to send an to Peter where's it location,
  • buy a cheap sensor we plug ourselves that's not cloud dependent.
    • if cloud at the mountain, imonnit - Peter think the existing base would work, just need new sensors.
    • if not, standalone solution
  • grab something from the imonnit he has already on the telescope? not the best
  • do we need outside acces to the internet (ASA)

• inventory sheet what went down, what remains to go down
• Pat's list - https://docs.google.com/spreadsheets/d/1-PUlL36oJZ1sMTrT435onpUL_zBpU5sswOtJaT-UYkk/edit#gid=0 - record when things go down
• 60F at the mountain CR

• anything needing physical presence to be turned off?
  • 1) soft shutdown everything
  • 2) turn off ports from PDU except the one we need - weather / tcs (needs to be confirmed)
  • 3) turn off the PDU
• coordination with mountain
  • use the all-hands open channel zoom + email to telescopeworks@lbto.org
  • call the mountain manager
  • Monday at 10AM, finalize things

• MODS - confirm when the shutdown really happens (Jerry still swapping fiber) _ Yang to send an email asking about the current plan
• obs computer go down after all the instruments have been brought down
• LBC - down (soft shutdown), Yang confirm by email it's all off.
• LUCI - Monday
• Deal with AO in a single big picture,
• once OVMS down, NAs, etc (pending obs1 resolution - in favor it goes down too, unless a clear use case)

Notes 2020-03-18

Systems remaining on

• webcams
  • web VM pulls data
    • migrate process to Tucson (SHooper)
    • wda with ENV and DDS to export real-time weather to info:// (pending tcs-se
• confirm for allsky
  • • it's handled by wda (SHopper)
• weatherstation
  • weather-pc
  • telemetry - TCS/ENV stays up ?
    • socket connection from TCS to weatherpc
    • use tcs-test, point it to the mountain (Yang, IT for network (mt-vlan on weatherpc) config and tucson-synha mount for hdf5)
      • use a separate location for the env telemetry hdf5 --- give dms a mount for last sample access
      • use script to push it to the "offial" tucson location

• core switch(es?) for network + 5 switches
  • PDU? problem to bring it back up
  • ~ 1.2 kW /per unit

• liebert UPS
  • keep CR telemetry ? * lightweight zabbix * power info, temperatures, not sure we have much to see
  • baremetal to snoop over the network bandwith

• big fan(no cooling)
  • but expected daily walk through inspection

Everything else shutdown in a safe state

• everything that produces heat that's not critical will go down

• check and update documentation along the way
• coordinate with instruments / mountain for workstations etc.
• keep OVMS up running until last
  • keep it running while "quiet" for a couple of hours
  • transfer telemetry for vibration analysis
• inventory sheet what went down

Miscellaneous

• opportunities and creative use of the shutdown?
  • allsky/vaisala, borrow spares for development?
  • xxx