UT 20130924 Science Observing

ISA: MEdwards (LBTO)
OSA: GBechetti (LBTO)
ISp: E Solheid

Summary

High IT (started critical): 4280 LUCI1: Compressor cold head broken High IT: 4281 LBCR tracker failure from PCS/IIF issues

Opened with LBC, clear skies, and good seeing. Ran a few superfoc sequences -- although some were contaminated with light (perhaps from local farms nearby?). Proceeded with OSURC science but ran into major dilemmas with LBCR tracker. See IT #4281. Called K. Summers who did some troubleshooting; finally determined after several hours (3+) that it was a PCS/IIF issue. Continued stress testing LCC as AdSec was being worked on by D. Miller. Took a number of "science/check" exposures with the moon up. They might be semi-valuable to the PIs:

• OSU_monitor_target1
• OSU_monitor_target2
• Martin_Field1
• Martin_Field2
• RB_rV_superfoc7.ob; RB_rV_superfoc180.ob

Ran-up LUCI only to discover CRITICAL state on instrument from high shield temps. Woke up Elliot, Werner, and Michael (both whom happened to be on the mountain for LUCI2). After some troubleshooting discovered compressor had failed. LUCI team set to work replacing LUCI1 compressor with LUCI2 compressor. Repair was successful and temperatures dropped extremely quickly! Should be ready for science tonight. Issue track still high, as compressor needs to be fixed, replaced to keep LUCI2 on schedule.

Ended the night with a few more LBC observations while instrument continued to cool.

Thanks to Kellee, Doug M and Doug S, the LUCI team, and Elliot for all their nighttime assistance!!!

Details

01:30 Olga taking data to check out MODS1 TCS fix

01:45 Opening shutters

01:50 Tried for twilight but too many stars really

02:00 MCS restart

02:15 DOFPIA on WT10_304; DIMM is reading about 0.9" @ El = 70 & X = 1.07

02:25 DOFPIA converged but at a bizarre seeing for blue (2"); running again to see if the second time around is more behaved

02:30 running co-pointing

02:40 Slewing to NGC101 for superfoc sequences --

02:45 Some interesting stray light here -- and we're awfully low. Geno closed the right rear vent door where we were seeing some stray light possibly from local farms. Confirmed 7 images in B and R for superfoc sequence though which is an improvement over last night! Hopefully the missing data problem/mystery was indeed solved

02:51 Slewing to ACT0287 for another superfoc sequence. DIMM is reading 0.8" @ X =1.01; El = 80

02:59 Slewing to OSU_monitor_target1focus.ob for dofpia; definitely noting that blue appers to have some flux not included in the pupils (skirts?). Red looks gorgeous.

03:07 Co-pointing target ACT_0382

03:15 Slewing to OSU_monitor_target1.ob

03:24 LBCR Tracker is stopped. RPC error? Not sure; don't see anything in the log??

03:31 No idea what that was about, Restarted the OB and everything seems fine. Don't see a log message corresponding to a warning? This one is a bit of a mystery.

03:37 Nope. At the very end of the exposure after 300s of guiding, red camera went into "Stop" again. No error messages. No readout of exposure. Turning lbc off and back on again! In the middle of reconnecting to LBT, the software actually "crashed". It did restart on it's own.

03:43 Trying OSU_monitor_target1.ob again

Ugh. Ok now: Image data is zero, chip1:0.000000, chip2:0.000000 [src/trackers/trackers.c:1660]

03:45 Same problem as before with stopped red tracker. Pulling system down completely

04:00 Images way out of focus. Doing FPIA again

04:12 Once more OSU_monitor_target1.ob. Same problem. After consulting with Mark and Olga, decide to call Kellee because right now the number of consistently working instruments is at a minimum. Clearly, John, I spoke much too soon

04:50 Trying OSU_monitor_target2focus.ob.

04:52 dofpia, /X2

05:00 Not converging in red. Perhaps too close to the moon or too low? Trying a different target from different project.

05:05 dofpia, /X2 on new field

05:15 OB! Same identical problem on 200s exposure. LBCR Tracker stops. Compared to Blue VERY FEW images being written to tec chip.

05:20 Decide to reboot everything and give Kellee another call. Result = same. Stalled LBCR. Kellee thinks that it might be time to try the older build of LBC software

05:30 Older build up and running. Still problems with LBCR

06:00 Starting to wonder if this isn't on PCS side. Have Geno restart PCS and IIF. Voila! Suddenly we start to have some success.

06:13 After consultation we head back to previous LBC build.

06:30 Another test. LBCR still behaving. Leaving build as is. Letting Kellee and Doug go with a hearty thanks!

06:40 Decide to stress test the instrument for the next hour. Data won't be solidly science with the moon up but want to convince everyone that this build is good and tracking is indeed working as it should be!

06:36 dofpia, /X2 on Martin_Field1

06:40 Observing Martin_Field1

07:08 SX AdSec e-mail from Doug. Not switching over to LUCI anytime soon apparently. Will keep at LBC stress testing with "science" data.

07:10 Slewing to Martin_Field2. Running dofpia

07:41 dofpia, /X2 on same field before superfoc sequence

07:47 RB_rV_superfoc7.ob @ El = 71; X = 1.057. Seeing ~ 1

lbcb.20130924.074754.fits lbcb.20130924.074841.fits lbcb.20130924.074931.fits lbcb.20130924.075020.fits lbcb.20130924.075111.fits lbcb.20130924.075200.fits lbcb.20130924.075250.fits

lbcr.20130924.074751.fits lbcr.20130924.074832.fits lbcr.20130924.074912.fits lbcr.20130924.074952.fits lbcr.20130924.075031.fits lbcr.20130924.075111.fits lbcr.20130924.075151.fits

07:56 RB_rV_superfoc7_pa180.ob

lbcb.20130924.075650.fits lbcb.20130924.075740.fits lbcb.20130924.075829.fits lbcb.20130924.075918.fits lbcb.20130924.080007.fits lbcb.20130924.080057.fits lbcb.20130924.080146.fits

lbcr.20130924.075646.fits lbcr.20130924.075727.fits lbcr.20130924.075808.fits lbcr.20130924.075848.fits lbcr.20130924.075928.fits lbcr.20130924.080007.fits lbcr.20130924.080049.fits

08:01 New target: Martin_Field3

08:12 oops. Forgot to stop OB and tried to run DOFPIA. Clearing the optics and starting DOFPIA again

08:26 checking LBC one last time after DOFPIA. Seeing is very good (0.7") and SURPRISE, we've got elongation on Blue. (082606)

08:28 Just got word from Doug that AdSec is now good to go.

08:33 Switching to LUCI; Running biases on LBC in the meantime

08:45 Critical LUCI temperature event. Internal shield temp is 77.8 Degrees and rising (albeit slowly now, it's leveling off) Calling D. Thompson and waking up Elliot.

09:25 Elliot finds Cold Head 1 off. Attempts to get it back on fail at first. Michael and Werner who are happily on the mountain find compressor is damaged. With Walter and John L. approval they decide to swap out the compressor with the LUCI2 compressor now on the mountain.

10:27 Switching back to LBC

10:49 Switch complete. Presetting to OSU_monitor_target3focus

10:50 too close to moon (as are most of the targets!) Trying for another, Martin_Field4. Again, likely not scientifically viable but may still be interesting and will be an adequate stress test to end the night.

11:00 dofpia on Martin_Field4. Computer issues from Tucson slow me down a bit -- need to logout of machine.

Exciting news from LUCI2 team -- compressor switch is done and compressor for LUCI1 is back up and running. Temperature has dropped extremely quickly -- looks like temps may stabilize in just an hour or so (instead of 10-12? Any explanation for why it's so fast?) Updated InformationTechnology and downgraded it to high.

11:10 Science on Martin_Field4 - field looks elongated. May have needed to redo co-pointing here, which I didn't quite get to it with the computer crash.

11:45 skyflats in LBCB (V) and LBCR (V). Which apparently take forever....

12:33 Adequate skyflats in LBCB (V) and LBCR (V)

-- MichelleEdwards - 24 Sep 2013