Staff:

LBTO:Josh, Andrew, Juan Carlos

Observer: Enrico, Guido, Fabio (remote Italy)

forecast.

forecast2_20230426.png

forecast0_20230426.png

forecast1_20230426.png

Summary.

Luci-SoulDX commissioning starts second half of the night.

Optical loop data stuck IT#8891

Stable seeing <1 arcsec. Good corrections ,optimal performances. Observations up to mag 17.4

Details.

11:54
  • Power on WFs.
  • change SW version: master-fwFix
  • Ocam
  • OK

04-27 07:27:47.902767| MAIN > Maximum mode index used to apply flat -> # 623


Thu Apr 27 07:38:16.765 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[2843] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0.0247677[0.0247677],roy:-0.0369566,mag:9.28,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Thu Apr 27 07:38:16.777 2023 aos.info - WFS source set to: FLAOWFS
  • CCD47 not live.
  • reset CCD47
  • ocam live.
  • cancel preset. there is not DIMM star and it is mandatory. (Enrico suggested to change star)
Thu Apr 27 07:51:02.251 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[2849] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0.168348[0.168348],roy:-0.0334582,mag:9.37,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
  • optimize:20230426_075550
  • TN:_075811,_075839,_075900,_075921,_075940,_080107,_080124
  • TN+Luci:_080353,_080449,_080526,_080557,_080632,_080708
ncpa on

521|DEB| 652|2023-04-27 08:07:51.304166| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 105.25, 0)
WfsArbitrator_128521|DEB| 653|2023-04-27 08:07:51.375025| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_128521|DEB| 654|2023-04-27 08:08:01.386045| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 105.29300000000001, 0)
WfsArbitrator_128521|DEB| 655|2023-04-27 08:08:01.399800| MAIN > arbFunc updateNcpaInternal done.
  • TN+Luc:_080802,_080828,_080907,_080937,_081007,_081037,_
offset 5 arcsec

luci darks.

1:16

stopAO

08:16:32.382955| MAIN > Maximum mode index used to apply flat -> # 237

AdSecArbitrator _31132|WAR| 1586|2023-04-27 08:16:27.811638| MAIN > Skip frame detected! (84.88% of frames skipped.)
AdSecArbitrator_31132|WAR| 1587|2023-04-27 08:16:28.919703| MAIN > Skip frame detected! (107.65% of frames skipped.)
AdSecArbitrator_31132|WAR| 1588|2023-04-27 08:16:28.919875| MAIN > Maximum skip frames reached: 107.65% of frames skipped. Will alert ICE clients.
AdSecArbitrator_31132|WAR| 1589|2023-04-27 08:16:29.924312| MAIN > Skip frame detected! (105.59% of frames skipped.)
AdSecArbitrator_31132|WAR| 1590|2023-04-27 08:16:29.924501| MAIN > Maximum skip frames reached: 105.59% of frames skipped. Will alert ICE clients.
AdSecArbitrator_31132|INF| 1591|2023-04-27 08:16:29.929302| MAIN > Received cmd: StopAo (1)

Thu Apr 27 08:20:06.630 2023 aos.PresetAO{FLAO}.complete - PresetAO {FLAO}[2856] - dx:nan dy:nan slNull: f1:0.0 f2:1.0 freq:231 mag:0 nBins:4 nModes:55 r0:nan snMode:0 strehl:0 ttMod:0
  • optimize:_082518
  • TN:_082855,_082916,_082937,_082957,_083017,_
  • loop open. 1693|2023-04-27 08:30:32.089032| MAIN > Skip frame detected! (0.72% of frames skipped.)
    20232704 08:30:37.9578 Alert: HighOrderOffload: HighOrderOffload vector ready
    AdSecArbitrator_31132|INF| 1694|2023-04-27 08:30:52.152815| COMMANDHANDLER > FSM (status: AORunning) has received command 2024 (Save status)
    AdSecArbitrator_31132|INF| 1695|2023-04-27 08:30:52.234346| COMMANDHANDLER > Command Save status (code 2024) successfully completed
    20232704 08:30:58.0038 Alert: HighOrderOffload: HighOrderOffload vector ready
    20232704 08:31:18.0364 Alert: HighOrderOffload: HighOrderOffload vector ready
    AdSecArbitrator _31132|WAR| 1696|2023-04-27 08:31:37.199017| MAIN > Skip frame detected! (108.99% of frames skipped.)
    AdSecArbitrator_31132|WAR| 1697|2023-04-27 08:31:37.199200| MAIN > Maximum skip frames reached: 108.99% of frames skipped. Will alert ICE clients.
    20232704 08:31:38.0849 Alert: HighOrderOffload: HighOrderOffload vector ready

loop open/reclose/optmize. This is not in the list of nudge.To investigate.
  • optimize: _083207
  • luci 5sec 10 frames
  • TN+Luci: _083352,_083513,_083640,_083754,_083907,_084017
  • offset 5 arcsec.
  • luci darks.
  • 20230427.0042,0043,0044
  • TN+Luci:_084755,_085109,_085224,_085342,_
pause for nudge

next nudge 09:24:42

optical loop diagnostic hung.

tLoopDiagnostic_57904|INF| 16|2023-04-27 08:57:23.683679 ARB-INTERFACE > ArbitratorInterface to adsecarb.R@M_ADSEC succesfully created!
OptLoopDiagnostic _57904|WAR| 17|2023-04-27 08:57:23.683722 GET_RAW_DATA > Error reading shared buffer: This operation requires previous initialization - trying to reattach [AdsecDiagnLib.cpp:14
  • the Stop AO doen;t goes through
  • hung.
  • open loop StopAO didn;t work.
  • fastlink still attached.
  • many error
  • it was forced to open the fastlink from the variable inspector.
  • cycle the arbitrators.
  • w-stop/start_all/
  • w_check:
-- Stage auxloops rewrite_gopt
startProcessByName(): starting process /home/AOeng/live/aoroot/bin/auxloops_soul.py -i auxloops
startProcessByName(): starting process /home/AOeng/live/aoroot/bin/rewrite_gopt.py -i rewrite_gopt
Traceback (most recent call last):
File "/home/AOeng/live/aoroot/bin/w_start", line 256, in <module>
if name == '__main__': main()
File "/home/AOeng/live/aoroot/bin/w_start", line 144, in main
processControl.waitProcessStart( app, process)
File "/home/AOeng/live/aoroot/lib/python/AdOpt/processControl.py", line 247, in waitProcessStart
raise AdOptError ( AOErrCode ['TIMEOUT_ERROR'], "Timeout error waiting for process %s start" % process)
AdOpt.AOExcept.AdOptError: Timeout error waiting for process rewrite_gopt start - TIMEOUT_ERROR
keepUp start/running, process 133340

w_check.

masterdiagnostic running and connected to Msgd
housekeeperWFS running and connected to Msgd
dbfiller running and connected to Msgd
wfsarb running and connected to Msgd
od_gopt running and connected to Msgd
gopt running and connected to Msgd
auxloops running and connected to Msgd
rewrite_gopt NOT running

  • optimize:_091702
  • TN:_092022,_092053,_092119,_092150,_
  • luci 5sec, frames 10
  • TN+Luci:_092241,_092506,_092622,_092741,_092850,_093048
02:26am

seeing:0.8"

pause nudge ->ok

next nudge: 09:58:50
  • offset 5 arcsec
  • luci darks.0059,0060
  • tn+luci:_093605,_093718,_093841,_094003,_094131,_094240
ncpaon
  • 09:44:01.288224| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 93.890199999999993, 0)
  • tn+luci:_094405,_094514,_094623,_094738,_094851,_095010
Thu Apr 27 09:52:15.735 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[2881] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0.0851576[0.0851576],roy:-0.0445763,mag:18.5,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)

optimize:
  • nudge during the optimize, but the loop remains closed.
  • optimize:_095918
  • tt:1.4,ho10.2
next nudge: 10:32:58
  • TN:_100301,_100335,_100411,_100452
  • TN+luci:_100522,_
3:13 all the software went down. Fabio investigating the issue offline

restart software and back again.

Preset again.

Thu Apr 27 10:17:07.520 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[2886] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0.057469[0.057469],roy:-0.0490022,mag:18.5,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
  • mag:17.7
  • optimize:_102244
  • luci10 frames 10
  • TN+luci:_102715,_102928,_103130,_
nudge.

0:32:42.0237 Alert: HighOrderOffload: HighOrderOffload vector ready
AdSecArbitrator_109000|WAR| 521|2023-04-27 10:32:59.683589| MAIN > Skip frame detected! (65.50% of frames skipped.)
AdSecArbitrator_109000|WAR| 522|2023-04-27 10:33:01.695964| MAIN > Skip frame detected! (113.45% of frames skipped.)
AdSecArbitrator_109000|WAR| 523|2023-04-27 10:33:01.696096| MAIN > Maximum skip frames reached: 113.45% of frames skipped. Will alert ICE clients.

taking background.

next nudge: 11:07:06

Thu Apr 27 10:40:53.268 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[2890] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:-0.032867[-0.032867],roy:-0.0434542,mag:14.17,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
  • optimize:_104504,_
  • TN:104630,_104646,_104658,_104712,_104727,_104751,_104830,_104849,_105013,_105037,_105113
  • luci exp 0.2 frames 30
  • TN+luci:_105232,_105309,_105341,_105418,_105443,_105509
  • offset 5"
  • luci background:0098,0099
  • ncpaon
  • B| 581|2023-04-27 10:57:57.754805| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 219.958, 0)
  • WfsArbitrator _30610|DEB| 582|2023-04-27 10:57:57.791005| MAIN > arbFunc updateNcpaInternal done.
  • TN+luci:_105815,_105844,_105912,_105938,_110006,_110032,_
ncpaoff

luci 2 frames 30
  • TN+luci:_110428,_
optical loop data stuck.
  • open fastlink.
  • restart wfsarbitrator.
  • force open loop from FLAO gui
  • restart optical loop
  • the gu is available
reclose/optimize
  • luci 2 sec frames 30
  • TN+luci:_111423,_111554,_111709,_111831,_
pupil rerotator not in trackng ...

back in tracking....
  • optimize:_112125
  • TN+Luci:_112246,_112434,_112601,_112742,_112902,_113031
  • offset 5 arc sec
  • luci background:0119,0120
  • tn+luci:_113547,_
next nudge: 11:41:14

Thu Apr 27 11:40:39.291 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[2906] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:-0.0625601[-0.0625601],roy:-0.0202734,mag:12.58,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)

next nudge: 12:15:22
  • optimize:_114423,_
  • double star
  • TN:_114719,_114737,_114828
change target

Thu Apr 27 11:48:43.207 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[2910] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:-0.0464964[-0.0464964],roy:-0.102702,mag:12,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
  • mag 11.7
  • optimize:_115304
  • TN:_115753,_115814,_115827,_115842,_115855,_115909,_115931,-120005,_120134,_120150,_
Luci2 has an error in filter#1 it is not reaching positions. not possible to take luci images.

no more time.

05:03am

StopAO

WFS set safe.

lucixx frames xx

TN+

nudge predictor:

0 start hour using backhour 12.0 end hour 12.0
These UT start times will have a nudge if the telescope is tracking.
00:18:34
00:52:42
01:26:50
02:00:58
02:35:06
03:09:14
03:43:22
04:17:30
04:51:38
05:25:46
05:59:54
06:34:02
07:08:10
07:42:18
08:16:26
08:50:34
09:24:42
09:58:50
10:32:58
11:07:06
11:41:14
12:15:22
12:49:30

-- JuanCarlosGuerra - 26 Apr 2023
Topic revision: r4 - 27 Apr 2023, JuanCarlosGuerra
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback