Restart Night #15 - UT 05 Sep 2020

OSA: J. Williams

ISp: C. Bolyard

Mtn Manager: T. Forsyth

SO: J. Power

AO: G. Taylor, B. Rothberg (scripting and targets)

1st half AO, 2nd half facility instruments

Night Center 00:19 - LUCI-->LBC, Handover 00:09


Started with OSCO which was completed with plenty of time to be on our fist target before 12 degree twillight. Obtained data on NGC6240, using the southern nucleus as the AOref. Data on PSF tars with OCAM magnitudes of 18.2 and 17.2 were obtained. Executed ESM/SL scripts using a bright AOref and Faint AOref. Discrepencies were noted between what OCAM is currently reporting as the magnitude of those reference stars and what was reported previously (now 1.5 mag fainter than catalog when previously ~0.5 mag). We switched to LBC's at night center and obtained rotator zero point data and a superfoc sequence on one field before a cold front produced large temperature gradients that made further data acquisition impossible. We finished hte night with LUCI Bino Spec, scripting large throws. the first acquisition had non sensical offsets calculated. The reason for that is unknown. A second acqusition was performed and everythign looked fine. On sky spectra looked good, although zJ spec may have a spectral shift (more detailed look needed). Temperature gradients only increased after we switched to LUCI so we made the right call.


Lost about an hour to WFS Adsec communications. Greg used the hammer restarting everything which got things back and in happy states.

A coldfront started to move in and generated a hiude temperature gradient for the second half. We could not collimate with LBC's except for the first half hours or so despite our best efforts.

IT 8176 - RB_Science kept crashing for Olga, Josh and myself with

XPA$ERROR no 'xpaset' access points match template: RB_Science
XPA$ERROR no 'xpaset' access points match template: RB_Science
XPA$ERROR no 'xpaset' access points match template: RB_Science
XPA$ERROR no 'xpaset' access points match template: RB_Science
XPA$ERROR no 'xpaget' access points match template: RB_Science

obs2 slowdowns. java 101%. Only running luci observer panel, alhtough I had the OT and other thigns up when the slwdowsn were noted earlier int eh night. It was unusable at times.

Proposed Plan:

  • Test new PCS patch (new from Feb 2020) using any observations that dither
  • LUCI/SOUL Checkout 1st half
    • DONENGC 6240 K' calibration/validation check (1hr)
      • close loop on southern nucleus
      • close look on off-axis star
    • DONEESM Verification, bright and faint AORef - K-band priority (1.5 hours)
    • Bin 4 followup - test for elongation detected previous night, verify R=18 - repeat at least one at this faint level
      • Need to take telemetry
    • Bin 1 subframe - bright star with subframe to 64 pixels use H-band, 200 frames per datacube - constrain the frequency of the jitter - 2 or 3 datasets
      • Need to take telemetry
    • Poor Weather Contingency
      • ESM Verification - bright and faint stars (only need to be able to close loop, poor weather helps)
  • Switch to Facility Instruments @00:09
  • LBC-Bino Checkout - hightest priority after AO
    • rotator center measurements for both LBCs
      • Doing this at 2-3 different elevations would be ideal
      • Slew to a copointing field and collimate and copoint
        • Any copointing script can be used. A jskycalc-formatted catalog of coords for the Sci-like OBs is in Restart_2020/LBC/ and a visibility plot is LBC_checkout_targets_vis.png
      • Run the series of OBs: ~/Service_Observing/Restart_2020/LBC/RotCen/pa0.ob, pa180.ob, pa90.ob and pa270
        • These OBs (paxx.ob) have been hacked to set DEC to -90 so that a new preset is not sent and the instrument is only rotated
      • collimation
      • super-focus sequence for both
        • superfoc at 2-3 different elevations would be ideal
  • DONELUCI-Bino Spectra quick OB (G210 H or K, G200 HK)
  • MODS-Bino GRB followup (spectroscopy if possible, otherwise imaging)


Precondition violation

1:40 Josh is opening. Unfortunately I had errors when i tried to save this afternoons log. I took some LBC and LUCI biases/darks. I had a precondition violation when initing LUCI2. zero's straing gauges resolved.

1:52 Slewing to OSCO V*SX He. Extra bright to the easst so changed my mind and slewed to HD230519 to jump on this a bit earlier.

2:00 Sky too bright. But I took a quick image and don't see the target. Josh is going to do a pointing check while we wait for the sky to dimm.

2:09 Preset to HD230519 successful. Seeing looks to be ~0.7"
  • OSCO image numbers luci1.20200905.0023-
  • Seeing variable though OSCO data collection, deteriorating at times to ~1.5". 0.6-1.6"
2:22 Slewing to NGC6240 on axis. Looks like center star selected the brighter of the 2 nuclei. But OCAM issue on center pupils.

2:32 Resending preset. OCAM not live on center pupil. Master Diagnostic process had crashed

2:27 Greg has recovered. Resending preset again.
  • luci1.20200905.0045-65
  • OCAM magnitude 17.0 centered on the southern nucleus (brighter more extended source), bin4 , frequency 198 , 54 modes.
  • luci1..0048. GCS star looked extended, perhaps both lobes were in th pupils or caused a jump?
  • Seeing ~0.7-1" from WFS on sky offset
3:27 Presetting to PSF R=17.01 mag
  • showing OCAM 18.2mag so may be a bit faint, bin 4 frequency 100, , 54 modes. ~18 average pupil intensity, 3.5 photons/subapp
  • luci1.20200905.0066-70
  • OVMS 120Hz is not large right now but there is aa large spike at 79Hz mostly on DX
  • psf looks elongated along diagonal
3:43 Presetting to PSF R=15.98
  • OCAM magnitude 17.3, bin4, frequency 137, 54 modes, ~30 average pupil intensity, 5.5 photons/subapp
  • Greg is taking lots fo telemetry data
  • luci1...0071-0075
  • Relatively round PSF
4:03 Josh has reauthorized as LUCI bino ansd we are presetting to M92 for ESM on LUCI1 and SL on LUCI2. AOref 9.4
  • Seeing 0.6- 0.7"
  • We got the wrong AO ref first try so Josh performed pointing check, preset back 4:16
  • Estimated magnitude 13.0? Barry has executed this before and has had the estimated magnitude closer to 10, so why is this coming out so much fainter? We are in bin2, frequency 100.
  • There is a script error. Hung on preset
  • Still OCAM showing an estimated magnitude of 13.0, bin2 frequency 100. ESM gains tt 5.36, ho1 3.12, ho2 0.1
  • Seeing 0.8" on the WFS
  • Very first sky offset Error copointing limit. So much for the PCS patch. Joch has disable the copointing patch.
  • OCAM appears frozen, Pupil check ocam2 process appears down. The process keeps dying. Greg is working to resolve that. Not closed for image luci1.20200905.0078. Looks like we need to warm the ocam and cycle it. (4:47)
  • luci1..0076- 78 luci2..0021-23
5:18 Greg has recovered the OCAM, resending presetting to M92 for ESM on LUCI1 and SL on LUCI2. AOref 9.4. Took about half an hour to recover.
  • luci1.20200905.0079-91, luci2.20200905.0024-36
  • OCAM reading 13.0 mag still, copointing still disabled. Bin2, frequency 100, 470 average pupil intensity, 86.9 average pupil intensity.
  • 0.8-0.9" on WFS, just under 0.48-0.5" on the ESM images and 0.65-0.7" SL
5:46 resending presetting to M92 for ESM on LUCI1 and SL on LUCI2. AOref 15.1.
  • OCAM reading 16.6 mag, previously was closer to 0.5 mag fainter than this is concerning.
  • ~70 average pupil intensity, 14.4 photons/subapp, bin4, frequency 100
  • looks like all the light is sitting in the upper right pupil only, even though loop closed. And we lost it. luci1.20200905.0095-0096.
  • Resending preset 5:58 failed again. WFS and Adsec are just not talking correctly, Greg is restarting the adsec
6:12 M92 is getting low so we are moving to M52, R15.2 ESM & SL
  • Rmag 16.8 on OCAM Average pupil intensity 55, 11.5 photons/subapp, bin4, frequency 100, modulation 6. (others up until now were modulation 3), gains 0.95, 0.65, 0
  • words like abomination are being thrown around to describe these mag differences
  • <0.4 on Luci1 and ~0.6 on luci2 and a bit elongated on luci2
  • luci1.20200905.0097-126, luci2.20200905.0042-71
7:10 Reconfiguring to LBC's

7:35 Copointing 20h60h
  • Olga is running dohybrid on first field. Huge defocus on LBCR needed, just like the other day. Red is calculating alot of Z8.
  • 7:47 Converged, co-pointing now.
7:58 Rotator Center PA=0, PA180, PA90, PA270at El=49
  • Rotator center measurement looks consistent with the past at first glance
obs2 was super laggy. moved my lbc firefox session locally (not running in x2go) and much! faster.

8:07 Recollimating before we run Superfoc

RB_Science keeps crashing for Olga, Josh and myself with

XPA$ERROR no 'xpaset' access points match template: RB_Science
XPA$ERROR no 'xpaset' access points match template: RB_Science
XPA$ERROR no 'xpaset' access points match template: RB_Science
XPA$ERROR no 'xpaset' access points match template: RB_Science
XPA$ERROR no 'xpaget' access points match template: RB_Science

messages. Olga has started to run in RB_science2 and Josh has stopped running RB_science hoping this would help stability, but the windows continue to crash repeatedly. (IT8176)

8:12 Running superFoc
  • 8:13 RB_rV_sf8 (PA0) el 47
  • 8:21 RB_rV_sf7_pa180 el 4r6
  • 8:28 RB_rv_sf7 (PA0) el 45, the seeing may be variable. Not a great fit to the superfoc sequence.
  • 8:37 Collimation does not look very good. Running ExtraIntra before we move to next field. RB_rVexin4 el=44
8:42 Collimation preset to 22h36d
  • running dofpia, blue converged with seeing of 0.89". Red ran away with spherical. cleared active optics and ran again.
  • runaway again, this time both sides in sperical. the primary is a degree warm. not sure why. Further investigation shows that the temps have been dropping slowly all night
Cleared active optics both sides, applied an initial offset to LBCR Z4
  • 9:11 Trying dofpia again, runaway
  • 9:25 we are clearing active optics, holding gains on Z11, Z22, and applying -7000 to DX Z4
  • Blue converged with 0.97" seeing, Red still failed to converge, Z4 runaway redsulting from the Z22 from teh temperature gradient
9:44 CoPointing and collimation 1h51d
  • holding gains on Z11, Z22, and applying -7000 to DX Z4. Blue converged, red did not (Z22 runaway? apparently gains were not held after all) but close enough. we are going to do the rotator center with this.
  • Running ExtraIntra focal pairs to help discern if there are any issues with Red. Red is quite out of focus, but we never did converge.
  • 10:19 Rotator Center - PA 0, 180, 90, 270. El 70
Looks terrible. We are not getting anythign useful with this temperature drop. It is not leveling off at all. We are going to reconfigure to LUCI

10:25. Josh is reconfiguring to LUCI's

10:47 Josh is doing a pointing check. We are going to execute a Bino Spectra OB at M76 with large throws, 0.75"LS zJ Spec, HKSpec

obs2 was being bogged down again. closing the OT seems to speed thigns up
  • Acquisition l1.20200905.0127-130, luci2 ..0072-0075
  • luci2 calculated a 6.2 degree offset to align the 2 stars int eh slit which is not correct. re trying while i bring up everything on obs3 (obs4 seems to have tsomeelses session and its frozen?)
11:23 Resending Preset to M76 for BinoSpec
  • Obs2 is unusable. Java is pegged at 101%. I'm only running luci observer panel!
  • luci1..131-133, luci2..0076-0078. Alignment looks much better, but was were the numbers so weird the first time?
  • luci1 0.5810, 0.0561, Rot 0.3191 Luci2 0.3781, -0.0580, Rot 0.2773 (not that the image used did not have an accurate PA)
  • Nice confirmation that aligning 2 stars along the slit procduced the same calculated rotator offset for LUCI1 and luci2
  • HK looks good. Sky lines ~2.7 FWHM in X on LUCI1 and 2.85 FWHM on LUCI2 on N1.8 images
  • luci1.20200905.00135-146. luci2.20200905.0080-91
12:09 Preset to telluric

12:13 Wrong telluric was selected so swinging back.
  • Telluric acquisition luci1..0147-149. luci2..0092-94
  • Failed instrument configuration grating when configuring grating - grating didnt stabilize
  • science luci1..0150-157. luci2..0095-102
  • zJ abosorption looks shifted?

Temperature gradients only increased after we switched to LUCI so we made the right call.

-- BarryRothberg - 04 Sep 2020
Topic revision: r8 - 05 Sep 2020, BarryRothberg
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback