UT 09-Feb-2020 SOUL Commissioning DX (SX backup)

  • Night Plan
    • Plan A:
      • Check out DX AdSec closed dome 1-2 hours (Arcentri team) - goal is to verify today's fix works
      • DX SOUL first light on-sky
      • Turn over at 1am local time to LBTO staff to continue either DX or SX SOUL data collection
      • SX/IRTC continue geo-sync satellite observations
    • Plan B (if DX AdSec problems continue or arise during night)
      • SX software, WFS, data collection with IRTC
      • Turn over at 1am local time to LBTO staff to continue SX SOUL data collection
      • SX/IRTC continue geo-sync satellite observations
    • Plan C (if no AO can be done)
      • D-time backup observations using available instruments (scripts for MODS-Bino and LUCI-2 available - LUCI-2 contingent on DX AdSec availability for SL)

OSA: J. Williams

ISp: E. Solheid

Mtn Manager: P. Hartley

ISA: B. Rothberg (Tucson)

AO: D. Miller (Tucson), Team Arcetri (Tucson), J. Christou (Tucson)

UT 00:50 - LUCI-2 init all - two test darks taken luci2.20200209.0001-0002.fits

UT 01:00 - RM 507-3 froze, couldn't access AO computers. Had to force an X kill (ssh'd into 507-3 and kill -9 the ID for Xorg) and Enrico was able to log back onto the desktop.

UT 01:27 - winds tapering downwards, Arcetri will work for another 30minutes closed dome and try to open

UT 01:33 - RM507-1 crashed again (vertical stripes on screen unable to remotely log in) - was planning to use this for sending presets and operating LUCI-2 and IRTC

irtc2.20200209.00001_cube.fits - test IRTC2 exposure - see it in newdata

UT 01:55 - DX AdSec failed, Arcetri on skype with Guido - software was restarted - discussion about whether to go up and reseat card or not

UT 02:05 - staying closed for now - leaning towards taking a look at AdSec while dome closed

UT 02:10 - proceeding with an intervention - contacting Pat and Elliott to head up to the adSec, meeting on Zoom2

-ran 2.5 hours with no problems, has been running about 30min since last failure

UT 02:37 - Pat and Elliott heading up

UT 03:11 - Pat and Elliott putting covers back in place after reseating electronics

UT 03:21 - RM 507-3 froze.

--had a LUCI-2 MOS error - robot was 20 steps short of N30field stop, recovered

UT 04:31 - preparing to open, winds in southwest

First Target AO 309 R=8.08, (H=6.2) GS=3 PA=0

if too close to the moon or need more north AO 326 R+8.7 (H=7.1) PA=0 GS=6

UT 04:44 - authorizing LUCI-2

UT 04:54 - holding off, we had a few red spikes (>20m/s) on the wind monitors

UT 05:16 - 20 minutes since series of gusts, planning to open enclosure

UT 05:21 - opening enclosure...

UT 05:25 - moving to pointing star

UT 05:31 - moving to collimation star

trouble colliating, ambient is -2, glass therm is 3.5

UT 05:39 - going to a R~9 pointing and collimation star on-axis to get started

UT 05:52 - hit the wind limit 20 and 22 m/s front and back - closing

UT 06:05 - going to try and open again

UT 06:15 - we are open - pointed north

UT 06:19 - difficulty collimating (kept hitting the hexapod limit)

UT 06:38 - closing

UT 07:09 - waiting....

UT 07:22 - opening

- going near AO 326 R+8.7 (H=7.1) PA=0 GS=6

altenrate is AO393 which is rising (GS 1 R=8.3 H=6.7)

UT 07:28 - pointing star

UT 07:29 - collimation star

-still hitting limits on secondary (RX)

UT 07:39 -zeroing out global offsets put in by Arcetri on DX

UT 07:48 - AO 326 HD 237711 K0III (R=8.7 (H=7.1) PA=0 GS=6)

-too much wind on adsec

-hit the red limits on wind

UT 07:51 - closing

Next to try: AO 445 GS4 R=8.1 H=6.0 HD=104675 K4III

UT 08:27 - opening dome - trying AO 445

UT 08:51 - sending collimation preset - had some trouble finding pointing star - high background? thin clouds?

UT 08:52 - AO 445 GS4 R=8.1 H=6.0 HD=104675 K4III

-didn't find guidestar

UT 08:55 - going to R=7 pointing star, don't see it

UT 08:59 - found the star

UT 08:59 - sending preset to AO 445

Found it!

UT 09:03 - sending runAO

-talking 2.51sec full frame FeII coord 918,1175

centered on 808,1047 with 256x256 sub-frame

UT 09:15 - taking data - luci2.20200209.0012.fits

UT 09:29 - shell RIP'd

UT 09:32 - sending preset to AO 468 GS=2 R=8.67 H=7.08 K0 star

-failed, loop stay closed (shouldn't a new preset open loop?)

UT 09:33 -sending again to AO 468

UT 09:35 - runAO

subframe 823 1035 256x256

-need Rotator to be 240, so need to find something 120 over (rotator is at 120 now - findng new star) - Arcetri says astigmatism from LUCI is causing image problems

UT 09:50 move to AO 464 PA=120 GS1 R=8.7 HD 121016

ended up at 248

star at 1103 1093 - use 975,965 256x256 - centered up

DIMM 1.08"

luci2.20200209.0025-0031, 0032-0033 background images

(Rotator Angle = PA - parallactic angle + offset in RT ZP)

UT 10:18 - AO989 GS=2 PA=150 parallactic = -90 now should get close to 240

-almost transiting, need to wait a bit (dearth of stars)

UT 10:33 -trailing into the wind...may need to move

UT 10:35 - AO 1004 PA=170 (parallactic -73) GS2

UT 10:41 - runAO

-arcetri wanted rotator=240 i got them 246 (not bad)

UT 10:46 - resending preset - AO problem so had to resend

-arbirtrator hung? seems stuck on PresetAO (AOS)

UT 10:48 -I'm baorting preset with IIF - didn't work
-brought up AOSGUI, they stopped preset AO and restarted AOS right, IIF got the failed notice, shell went to RIP, then rested (because no elevation info)

-stop and restart adsec arbitrator, diong this again

UT 10:52 - stop and restart adsec arbitrator, diong this again

IDL appears to be stuck doing something - this prevents any IDL GUIs from doing anything until process completes

need to do Adsec Stop/Start (Doug's information)

UT 10:59 - recovered AO, need to clear active optics on the primary (need to stop sending zernikes to primary for a bit wait for SH to update, then turn back on)

UT 11:01 - resending preset to AO 1004


UT 11:04 - resending preset to AO 1004

UT 11:14 - taking data 0034-0039, bkgd 0040-0041

subframe 845,972

UT 11:25 - AO 521 GS=1 parallactic -83 PA=150 GS=1 R=13.2 H=9.5

-colliimation not great, took a while, sending runAO

UT 11:31 - shell RIP

UT 11:32 - seem to be stuck in an acquireREfAO (AOS)

UT 11:35 - resending preset to AO 521


- switched to just H-band

-received DX AdSec error warnings

UT 11:44 - centering fieldstop

UT 11:56 - waiting for AO loop to close

UT 11:58 - still unable to close loop

UT 12:10 - AO590 GS=4 PA=165 parallactic -71 now R=9.8 H=7.9

-trying slightly brighter to see if same behavior with R=13 star

Error seen recently are in actuators which are also in Crate 1 (BCU)

-if see same errors, Doug recommends stop and restarting AdSec

UT 12:12 - preset to AO 590 failed - powerboard.r.reset_bcu_47. timeout error

-switcing back to FeII filter

UT 12:18 - sending preset to AO 590 again


UT 12:24 -same issue with actuator

-Rest, power off, AdSec Stop, Start

UT 12:28 - resending preset to AO 590 again

UT 12:30 - seeing around 1.8-1.9" on DIMM
UT 12;35 - DIMM 2"

UT 12:39 - still having issues, skip frames, etc - AdSec?

-seem to be high forces on several acuators

UT 12:43 - SOUL team is giving up as closing the loop is not working, but Doug will try removing an actuator to see if it improves anything

LBTO staff now debugging DX issues...

UT 12:49 - sending preset to AO 590 again

- failed because still stuck in PresetAO

-Actuator 112 and 115 bad? Doug removing 112

UT 13:03 - stopping and restarting DX AdSec

Note: The DX Soul software is not complete - things like RunAO and closing the loop still has to be done manually, the DX version of the software does not work like the SX version.

Five Steps when do runAO, 2 of the five do not work properly. Without manual intervention center pupils and close loops locks itself into an infinite loop.

UT 13:13 - making another attempt with actuator 112 removed, need some time to clear active optics on primary and allow SH spots to not send crud

-reasonably collimated, sending runAO

UT 13:22 - putting LUCI-2 in safe state

UT 13:23 - taking out actuator 116

-returned error to IDL prompt

-turning off zernikes to the primary

-stopping restarting Adsec

now have 640 modes

UT 13:31 - resending preset to AO 590


-loop cloed but ADSEC looks pinched

UT 13:37- Doug is done....leaving this to Guido

The adventure continues....

-- BarryRothberg - 09 Feb 2020
Topic revision: r2 - 09 Feb 2020, BarryRothberg
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback