Summary.

https://docs.google.com/spreadsheets/d/1jjr8oyWWlJs5v0xp1_9LHLynhbMaVjfZDHVC2gEC7sQ/edit#gid=1058287513

Details.

Currently running: stable (soul)
Available versions in /home/AOeng/soul:
stable
stable.20221117
stable.20230201
stable.20230202
test.20230302.offsetZ
test.20230310.ncpa-threshold
test.20230322.ncpa-threshold-and-fwFix
test.20230322.rerotator-and-fwFix
[AOeng@soul-sxwfs source]$ use_soul test.20230322.ncpa-threshold-and-fwFix
Switched live version to /home/AOeng/soul/test.20230322.ncpa-threshold-and-fwFix.
[AOeng@soul-sxwfs source]$ use_soul
Usage: use_soul [version]
Currently running: test.20230322.ncpa-threshold-and-fwFix (soul)
Available versions in /home/AOeng/soul:
stable
stable.20221117
stable.20230201
stable.20230202
test.20230302.offsetZ
test.20230310.ncpa-threshold
test.20230322.ncpa-threshold-and-fwFix
test.20230322.rerotator-and-fwFix

10:50
  • power WFS on
  • ocam ok
  • Adsec flat.
  • 17:45:45.928251| MAIN > Maximum mode index used to apply flat -> # 639

test preset.

Wed Mar 22 17:56:57.892 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[10] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 17:56:57.909 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 17:56:57.975 2023 aos.info - FLAO: WARNING: added atmospheric disturbance file dist_flao2_KL_v20_atm_s1.5_L040.0_v15.0_ovfreq500.0000_sd3892.fits

Wed Mar 22 18:01:37.537 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[12] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 18:01:37.553 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 18:01:37.593 2023 aos.info - FLAO: WARNING: added atmospheric disturbance file dist_flao2_KL_v20_atm_s1.6_L040.0_v15.0_ovfreq1700.00_sd3892.fits

  • optimize:20230322_180821
gopton
  • run the reset_gopt_forever.sh
  • TN:_181213,_181229

11:13 NCAP track on
  • tracking at 1sec.
  • stopAO
  • Brandon debug.
11:27
  • restart WFS arbitrator.
Wed Mar 22 18:18:38.725 2023 aos.Stop{FLAO}.complete - Stop{FLAO}[15]
Wed Mar 22 18:27:38.503 2023 aos.info - Updated variable AOARB.L.RR_ENABLED: 1
Wed Mar 22 18:27:43.357 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[16] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 18:27:43.372 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 18:27:43.429 2023 aos.info - FLAO: WARNING: added atmospheric disturbance file dist_flao2_KL_v20_atm_s1.6_L040.0_v15.0_ovfreq1700.00_sd3892.fits
  • optimize:183122
  • gopt
  • TN+luci:183353
  • error saving optical lop data
  • TN:_183435,_183455
  • ncpa on
216|DEB| 208|2023-03-22 18:35:50.217654| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_106216|DEB| 209|2023-03-22 18:35:51.219093| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_106216|DEB| 210|2023-03-22 18:35:51.219565| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=4, minimum interval=10)
WfsArbitrator_106216|DEB| 211|2023-03-22 18:35:51.219851| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_106216|DEB| 212|2023-03-22 18:35:52.221095| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_106216|DEB| 213|2023-03-22 18:35:52.221496| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=5, minimum interval=10)
WfsArbitrator_106216|DEB| 214|2023-03-22 18:35:52.221758| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_106216|DEB| 215|2023-03-22 18:35:53.223124| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_106216|DEB| 216|2023-03-22 18:35:53.223447| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=6, minimum interval=10)
WfsArbitrator_106216|DEB| 217|2023-03-22 18:35:53.223640| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_106216|DEB| 218|2023-03-22 18:35:54.224770| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_106216|DEB| 219|2023-03-22 18:35:54.225168| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=7, minimum interval=10)
WfsArbitrator_106216|DEB| 220|2023-03-22 18:35:54.225360| MAIN > arbFunc updateNcpaInternal done.
  • NCPAON
  • Adec received vector
18:35:47.332612| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,-19.34e-9,-15.65e-9,28.11e-9,-0.28e-9,8.12e-9,-7.52e-9,-7.55e-9,-3.69e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI

18:37:47.761047| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,-19.34e-9,-15.65e-9,28.11e-9,-0.28e-9,8.12e-9,-7.52e-9,-7.55e-9,-3.69e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI
idl_ctrl_54614 |INF| 25393|2023-03-22 18:37:47.761193| MAIN > StartCmd
idl_ctrl_54614 |INF| 25394|2023-03-22 18:37:47.767077| MAIN > EndCmd
idl_ctrl_54614 |INF| 25395|2023-03-22 18:37:48.426571| MAIN > Executing command: print,test_demo_mode()
idl_ctrl_54614 |INF| 25396|2023-03-22 18:37:48.426759| MAIN > StartCmd
idl_ctrl_54614 |INF| 25397|2023-03-22 18:37:48.426806| MAIN > IDL license file is running.
idl_ctrl_54614 |INF| 25397|2023-03-22 18:37:48.426806| MAIN > . 0
idl_ctrl_54614 |INF| 25397|2023-03-22 18:37:48.426806| MAIN > . EndCmd
idl_ctrl_54614 |INF| 25398|2023-03-22 18:37:57.860614| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,-19.34e-9,-15.65e-9,28.11e-9,-0.28e-9,8.12e-9,-7.52e-9,-7.55e-9,-3.69e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI

11:39 ocam not live during close loop
  • adsec failure
  • stopAO
11:43
  • adsc-stop/start WFS_stop/start
  • flat
  • 18:50:21.237696| MAIN > Maximum mode index used to apply flat -> # 639
Wed Mar 22 18:53:49.557 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[20] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 18:53:49.573 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 18:53:49.625 2023 aos.info - FLAO: WARNING: added atmospheric disturbance file dist_flao2_KL_v20_atm_s1.6_L040.0_v15.0_ovfreq1700.00_sd3892.fits

  • optimize:_185845
  • skip frames up to 60%
  • ocam not live when the loop was closed during optimize gain
  • gopt
  • power cycle bcu2k and rebin 1x1 vcropped.
Wed Mar 22 19:06:18.567 2023 aos.AcquireRef{FLAO}.complete - AcquireRef {FLAO}[21] - dx:-1.2049 dy:-0.0866761 slNull: f1:0.0 f2:1.0 freq:1700 mag:0 nBins:1 nModes:501 r0:nan snMode:0 strehl:0 ttMod:0
Wed Mar 22 19:06:24.651 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[22] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 19:06:24.666 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 19:06:24.716 2023 aos.info - FLAO: WARNING: added atmospheric disturbance file dist_flao2_KL_v20_atm_s1.6_L040.0_v15.0_ovfreq1700.00_sd3892.fits

WfsArbitrator _16939|INF| 383|2023-03-22 19:08:12.143345| MAIN > calibHODarkInternal(): done
WfsArbitrator_16939|DEB| 384|2023-03-22 19:08:23.191873| MAIN > arbFunc doCenterPupils done.
WfsArbitrator_16939|DEB| 385|2023-03-22 19:08:23.192836| MAIN > arbFunc doCloseLoop (, 'FLAOWFS', 'LUCIFER')
WfsArbitrator_16939|DEB| 386|2023-03-22 19:08:23.194688| MAIN > arbFunc doCloseLoop done.
WfsArbitrator_16938|WAR| 270|2023-03-22 19:08:24.880932| MAIN > OCAM Bad Frames/Slopes Warning, code 128
WfsArbitrator_16938|WAR| 271|2023-03-22 19:08:24.880979| MAIN > OCAM Warning Description: Code 128 - Noisy Octants: Pixel RMS in external octants is high
WfsArbitrator_16938|WAR| 272|2023-03-22 19:08:24.880993| MAIN > OCAM Suggested action to recover: No action required; no impact on performance is expected
20232203 19:08:24.8877 Warning: Warning state caused by Bad Frames or Slopes, CODE:128 - No Action required

  • close loop
  • ocam not live
  • live when loop opens
  • 12:10
  • power cycle bcu2k

Wed Mar 22 19:13:18.382 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[25] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 19:13:18.397 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 19:13:18.448 2023 aos.info - FLAO: WARNING: added atmospheric disturbance file dist_flao2_KL_v20_atm_s1.6_L040.0_v15.0_ovfreq1700.00_sd3892.fits
  • optimize_191715
  • gopt
  • TN:_192006,_192035,_
  • 12:21
  • ncpa on
2023-03-22 19:21:20.942569| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,-19.34e-9,-15.65e-9,28.11e-9,-0.28e-9,8.12e-9,-7.52e-9,-7.55e-9,-3.69e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI
idl_ctrl_126283 |INF| 6718|2023-03-22 19:21:20.942658| MAIN > StartCmd
idl_ctrl_126283 |INF| 6719|2023-03-22 19:21:20.948160| MAIN > EndCmd
idl_ctrl_126283 |INF| 6720|2023-03-22 19:21:24.012943| MAIN > Executing command: print,test_demo_mode()
idl_ctrl_126283 |INF| 6721|2023-03-22 19:21:24.013063| MAIN > StartCmd
idl_ctrl_126283 |INF| 6722|2023-03-22 19:21:24.013080| MAIN > IDL license file is running.
idl_ctrl_126283 |INF| 6722|2023-03-22 19:21:24.013080| MAIN > . 0
idl_ctrl_126283 |INF| 6722|2023-03-22 19:21:24.013080| MAIN > . EndCmd
  • TN+luci:_no saved.
  • 19:22:24.013485| MAIN > . EndCmd
    idl_ctrl_126283 |INF| 6726|2023-03-22 19:23:12.911542| MAIN > Executing command: errcode= fsm_savestatus('/local/aomeas/adsec_data/20230322/Data_20230322_192312/adsec.sav') & print, errcode
    idl_ctrl_126283 |INF| 6727|2023-03-22 19:23:12.911705| MAIN > StartCmd
    idl_ctrl_126283 |INF| 6728|2023-03-22 19:23:13.004953| MAIN > 0
    idl_ctrl_126283 |INF| 6729|2023-03-22 19:23:13.004981| MAIN > EndCmd
    idl_ctrl_126283 |INF| 6730|2023-03-22 19:23:24.013723| MAIN > Executing command: print,test_demo_mode()
    idl_ctrl_126283 |INF| 6731|2023-03-22 19:23:24.013911| MAIN > StartCmd
    idl_ctrl_126283 |INF| 6732|2023-03-22 19:23:24.013935| MAIN > IDL license file is running.
    idl_ctrl_126283 |INF| 6732|2023-03-22 19:23:24.013935| MAIN > . 0
    idl_ctrl_126283 |INF| 6732|2023-03-22 19:23:24.013935| MAIN > . EndCmd
    idl_ctrl_126283 |INF| 6733|2023-03-22 19:23:31.685019| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,-19.34e-9,-15.65e-9,28.11e-9,-0.28e-9,8.12e-9,-7.52e-9,-7.55e-9,-3.69e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI
    idl_ctrl_126283 |INF| 6734|2023-03-22 19:23:31.685169| MAIN > StartCmd
    idl_ctrl_126283 |INF| 6735|2023-03-22 19:23:31.691138| MAIN > EndCmd
    idl_ctrl_126283 |INF| 6736|2023-03-22 19:23:41.725726| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,-19.34e-9,-15.65e-9,28.11e-9,-0.28e-9,8.12e-9,-7.52e-9,-7.55e-9,-3.69e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI
    idl_ctrl_126283 |INF| 6737|2023-03-22 19:23:41.725849| MAIN > StartCmd

  • TRACk off -03-22 19:24:24.014373| MAIN > . EndCmd
    idl_ctrl_126283 |INF| 6742|2023-03-22 19:24:43.879001| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI
    idl_ctrl_126283 |INF| 6743|2023-03-22 19:24:43.879125| MAIN > StartCmd
    idl_ctrl_126283 |INF| 6744|2023-03-22 19:24:43.885099| MAIN > EndCmd
    idl_ctrl_126283 |INF| 6745|2023-03-22 19:24:43.885118| MAIN > % Program caused arithmetic error: Floating illegal operand
    idl_ctrl_126283 |INF| 6745|2023-03-22 19:24:43.885118| MAIN > . % Detected at Z2M4ADSEC 99 /home/aoeng/live/aoroot/idl/adsec_lib/adsec
    idl_ctrl_126283 |INF| 6745|2023-03-22 19:24:43.885118| MAIN > . _utilities/z2m4adsec.pro

12:25
  • track on
1311|2023-03-22 19:24:43.750247| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 0, 1)
WfsArbitrator_16939|DEB| 1312|2023-03-22 19:24:43.750593| MAIN > Reset NCPA interval counter and previous NCPA vector.
WfsArbitrator_16939|DEB| 1313|2023-03-22 19:24:43.758061| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_16939|DEB| 1314|2023-03-22 19:25:29.874017| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_16939|DEB| 1315|2023-03-22 19:25:29.874362| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=1, minimum interval=10)
WfsArbitrator_16939|DEB| 1316|2023-03-22 19:25:29.874508| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_16939|DEB| 1317|2023-03-22 19:25:30.875932| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.0

23-03-22 19:25:24.014731| MAIN > . EndCmd
idl_ctrl_126283 |INF| 6750|2023-03-22 19:25:39.037443| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,-19.34e-9,-15.65e-9,28.11e-9,-0.28e-9,8.12e-9,-7.52e-9,-7.55e-9,-3.69e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI
idl_ctrl_126283 |INF| 6751|2023-03-22 19:25:39.037569| MAIN > StartCmd
idl_ctrl_126283 |INF| 6752|2023-03-22 19:25:39.043004| MAIN > EndCmd
  • save track number.
23-03-22 19:25:24.014731| MAIN > . EndCmd
idl_ctrl_126283 |INF| 6750|2023-03-22 19:25:39.037443| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,-19.34e-9,-15.65e-9,28.11e-9,-0.28e-9,8.12e-9,-7.52e-9,-7.55e-9,-3.69e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI
idl_ctrl_126283 |INF| 6751|2023-03-22 19:25:39.037569| MAIN > StartCmd
idl_ctrl_126283 |INF| 6752|2023-03-22 19:25:39.043004| MAIN > EndCmd
  • not more updates of Z0.
  • TN+Luci
03-22 19:26:24.015054| MAIN > . EndCmd
idl_ctrl_126283 |INF| 6761|2023-03-22 19:27:16.158465| MAIN > Executing command: errcode= fsm_savestatus('/local/aomeas/adsec_data/20230322/Data_20230322_192716/adsec.sav') & print, errcode
idl_ctrl_126283 |INF| 6762|2023-03-22 19:27:16.158594| MAIN > StartCmd
idl_ctrl_126283 |INF| 6763|2023-03-22 19:27:16.250655| MAIN > 0
  • optical loop data time out
  • not more updates.
  • manual cube: 20230323.0001
WfsArbitrator _16939|DEB| 4085|2023-03-22 19:40:22.279632| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator _16939|DEB| 4086|2023-03-22 19:40:22.290779| MAIN > Loaded NCPA for rotAngle=341.002000, thresholds: [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
WfsArbitrator_16939|DEB| 4087|2023-03-22 19:40:22.290894| MAIN > Ignored NPCA because the change is less than the threshold for one or more modes.
WfsArbitrator_16939|DEB| 4088|2023-03-22 19:40:22.291164| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_16939|DEB| 4089|2023-03-22 19:40:23.292664| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_16939|DEB| 4090|2023-03-22 19:40:23.293114| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=1, minimum interval=10)
WfsArbitrator_16939|DEB| 4091|2023-03-22 19:40:23.293294| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_16939|DEB| 4092|2023-03-22 19:40:24.294710| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_16939|DEB| 4093|2023-03-22 19:40:24.295106| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=2, minimum interval=10)
WfsArbitrator_16939|DEB| 4094|2023-03-22 19:40:24.295280| MAIN > arbFunc updateNcpaInternal done.
  • restart optical loop diagnostic process.
  • try grab image s from gui
  • and it reports and error.
r_16939|INF| 6520|2023-03-22 19:55:43.139473| MAIN > Exception: list index out of range
WfsArbitrator_16939|ERR| 6521|2023-03-22 19:55:43.139658| MAIN > RETRY: Cannot save PSF (0) NO_ERROR

12:53

StopAO
  • wfsarb stop/start
Wed Mar 22 19:56:34.070 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[29] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
  • AOs is hung with the preset.
  • Brandon debug. Some type error
Wed Mar 22 20:02:41.227 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[31] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 20:02:41.252 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 20:02:41.966 2023 aos.info - FLAO: WARNING: added atmospheric disturbance file dist_flao2_KL_v20_atm_s1.6_L040.0_v15.0_ovfreq1700.00_sd3892.fits
  • optimize_200555,
  • gopt
  • TN:_200751,200820
  • ncpa on
23-03-22 20:08:22.022997| MAIN > getSetupFits(): fits header created
WfsArbitrator_9735 |DEB| 189|2023-03-22 20:08:22.062843| MAIN > arbFunc doSaveStatus done.
WfsArbitrator_9735 |DEB| 190|2023-03-22 20:09:20.374862| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_9735 |DEB| 191|2023-03-22 20:09:20.375244| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=1, minimum interval=10)
WfsArbitrator_9735 |DEB| 192|2023-03-22 20:09:20.375476| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_9735 |DEB| 193|2023-03-22 20:09:21.376808| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_9735 |DEB| 194|2023-03-22 20:09:21.377179| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=2, minimum interval=10)
WfsArbitrator_9735 |DEB| 195|2023-03-22 20:09:21.377372| MAIN > arbFunc updateNcpaInternal done.

22|2023-03-22 20:09:29.555098| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,-0.00e-9,-0.00e-9,0.00e-9,-0.00e-9,0.00e-9,-0.00e-9,-0.00e-9,-0.00e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI
idl_ctrl_126283 |INF| 9823|2023-03-22 20:09:29.556643| MAIN > StartCmd
idl_ctrl_126283 |INF| 9824|2023-03-22 20:09:29.562355| MAIN > EndCmd
idl_ctrl_126283 |INF| 9825|2023-03-22 20:09:29.562401| MAIN > % Program caused arithmetic error: Floating illegal operand
idl_ctrl_126283 |INF| 9825|2023-03-22 20:09:29.562401| MAIN > . % Detected at Z2M4ADSEC 99 /home/aoeng/live/aoroot/idl/adsec_lib/adsec
idl_ctrl_126283 |INF| 9825|2023-03-22 20:09:29.562401| MAIN > . _utilities/z2m4adsec.p

bitrator_9735 |DEB| 329|2023-03-22 20:09:59.560346| MAIN > -1.56500000e-08 2.81100000e-08 -2.80000000e-10 8.12000000e-09
WfsArbitrator_9735 |DEB| 330|2023-03-22 20:09:59.560380| MAIN > -7.52000000e-09 -7.55000000e-09 -3.69000000e-09]
WfsArbitrator_9735 |DEB| 331|2023-03-22 20:09:59.560462| MAIN > Ignored NPCA because the change is less than the threshold for one or more modes.
WfsArbitrator_9735 |DEB| 332|2023-03-22 20:09:59.560736| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_9735 |DEB| 333|2023-03-22 20:10:00.562094| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_9735 |DEB| 334|2023-03-22 20:10:00.562527| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=1, minimum interval=10)
WfsArbitrator_9735 |DEB| 335|2023-03-22 20:10:00.562728| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_9735 |DEB| 336|2023-03-22 20:10:01.564091| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_9735 |DEB| 337|2023-03-22 20:10:01.564509| MAIN > Ignored NCPA because there has not been enough time sinc

....

_9735 |DEB| 862|2023-03-22 20:12:29.153304| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=9, minimum interval=10)
WfsArbitrator_9735 |DEB| 863|2023-03-22 20:12:29.153416| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_9735 |DEB| 864|2023-03-22 20:12:30.154881| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_9735 |DEB| 865|2023-03-22 20:12:30.166293| MAIN > Loaded NCPA for rotAngle=341.002000, values=[ 0. 0. 0. -19.33685493 -15.64848518
WfsArbitrator_9735 |DEB| 866|2023-03-22 20:12:30.166367| MAIN > 28.10965729 -0.28022152 8.1203022 -7.52223873 -7.54709625
WfsArbitrator_9735 |DEB| 867|2023-03-22 20:12:30.166394| MAIN > -3.68989944], thresholds: [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
WfsArbitrator_9735 |DEB| 868|2023-03-22 20:12:30.167326| MAIN > Truncating ncpa vector to new values: [ 0.00000000e+00 0.00000000e+00 0.00000000e+00 -1.93400000e-08
WfsArbitrator_9735 |DEB| 869|2023-03-22 20:12:30.167366| MAIN > -1.56500000e-08 2.81100000e-08 -2.80000000e-10 8.12000000e-09
WfsArbitrator_9735 |DEB| 870|2023-03-22 20:12:30.167388| MAIN > -7.52000000e-09 -7.55000000e-09 -3.69000000e-09]
WfsArbitrator_9735 |DEB| 871|2023-03-22 20:12:30.167459| MAIN > Ignored NPCA because the change is less than the threshold for one or more modes.
WfsArbitrator_9735 |DEB| 872|2023-03-22 20:12:30.167686| MAIN > arbFunc updateNcpaInternal done.

  • stopao
  • Brandon debug the zo of zeros....
1:16
  • restart wfsarb
Wed Mar 22 20:16:40.839 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[35] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 20:16:40.857 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 20:16:40.927 2023 aos.info - FLAO: WARNING: added atmospheric disturbance file dist_flao2_KL_v20_atm_s1.6_L040.0_v15.0_ovfreq1700.00_sd3892.fits
  • optimize:_202036
  • gopt.
  • TN:_202231,_202251
  • ncpa on
-03-22 20:23:25.707204| MAIN > 0
idl_ctrl_126283 |INF| 12597|2023-03-22 20:23:25.707204| MAIN > . EndCmd
idl_ctrl_126283 |INF| 12598|2023-03-22 20:23:32.743604| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,-19.34e-9,-15.65e-9,28.11e-9,-0.28e-9,8.12e-9,-7.52e-9,-7.55e-9,-3.69e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI
idl_ctrl_126283 |INF| 12599|2023-03-22 20:23:32.744911| MAIN > StartCmd
idl_ctrl_126283 |INF| 12600|2023-03-22 20:23:32.750701| MAIN > EndCmd

.....

r_84989|DEB| 391|2023-03-22 20:24:22.800177| MAIN > 28.10965729 -0.28022152 8.1203022 -7.52223873 -7.54709625
WfsArbitrator_84989|DEB| 392|2023-03-22 20:24:22.800200| MAIN > -3.68989944], thresholds: [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
WfsArbitrator_84989|DEB| 393|2023-03-22 20:24:22.801297| MAIN > Truncating ncpa vector to new values: [ 0. 0. 0. -19.34 -15.65 28.11 -0.28 8.12 -7.52 -7.55
WfsArbitrator_84989|DEB| 394|2023-03-22 20:24:22.801360| MAIN > -3.69]
WfsArbitrator_84989|DEB| 395|2023-03-22 20:24:22.801439| MAIN > Ignored NPCA because the change is less than the threshold for one or more modes.
WfsArbitrator_84989|DEB| 396|2023-03-22 20:24:22.801802| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_84989|DEB| 397|2023-03-22 20:24:23.803148| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_84989|DEB| 398|2023-03-22 20:24:23.803468| MAIN > Ignored NCPA because there has not been enough time since the last update. (counter=1, minimum interval=10)
WfsArbitrator_84989|DEB| 399|2023-03-22 20:24:23.803621| MAIN > arbFunc updateNcpaInternal done.
WfsArbitrator_84989|DEB| 400|2023-03-22 20:24:24.804797| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 341.00200000000001, 0)
WfsArbitrator_84989|DEB| 401|2023-03-22 20:24:24.805136| MAIN > Ignored NCPA because there has not been enough time

9388| MAIN > Loaded NCPA for rotAngle=341.002000, values=[ 0. 0. 0. -19.33685493 -15.64848518
WfsArbitrator_84989|DEB| 531|2023-03-22 20:25:02.969451| MAIN > 28.10965729 -0.28022152 8.1203022 -7.52223873 -7.54709625
WfsArbitrator_84989|DEB| 532|2023-03-22 20:25:02.969488| MAIN > -3.68989944], thresholds: [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
WfsArbitrator_84989|DEB| 533|2023-03-22 20:25:02.971188| MAIN > Truncating ncpa vector to new values: [ 0. 0. 0. -19.34 -15.65 28.11 -0.28 8.12 -7.52 -7.55
WfsArbitrator_84989|DEB| 534|2023-03-22 20:25:02.971241| MAIN > -3.69]
WfsArbitrator_84989|DEB| 535|2023-03-22 20:25:02.971346| MAIN > Ignored NPCA because the change is less than the threshold for one or more modes.
WfsArbitrator_84989|DEB| 536|2023-03-22 20:25:02.971646| MAIN > arbFunc updateNcpaInternal done.
  • TN+luci. no luci saved, erro the optical loop data
01:34
  • ncpa off
WfsArbitrator _84989|DEB| 2434|2023-03-22 20:34:05.753863| MAIN > arbFunc updateNcpaInternal (, 'FLAOWFS', 'LUCIFER', 0, 1)
WfsArbitrator_84989|DEB| 2435|2023-03-22 20:34:05.754278| MAIN > Reset NCPA interval counter and previous NCPA vector.
WfsArbitrator_84989|DEB| 2436|2023-03-22 20:34:05.761232| MAIN > arbFunc updateNcpaInternal done.

3-22 20:34:05.883103| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9,0.00e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI
idl_ctrl_126283 |INF| 12641|2023-03-22 20:34:05.883247| MAIN > StartCmd
idl_ctrl_126283 |INF| 12642|2023-03-22 20:34:05.888537| MAIN > EndCmd
idl_ctrl_126283 |INF| 12643|2023-03-22 20:34:05.888580| MAIN > % Program caused arithmetic error: Floating illegal operand
idl_ctrl_126283 |INF| 12643|2023-03-22 20:34:05.888580| MAIN > . % Detected at Z2M4ADSEC 99 /home/aoeng/live/aoroot/idl/adsec_lib/adsec
idl_ctrl_126283 |INF| 12643|2023-03-22 20:34:05.888580| MAIN > . _utilities/z2m4adsec.

1:35
  • ncpa on
3-03-22 20:35:24.573406| MAIN > Executing command: set_z0, [0.00e-9,0.00e-9,0.00e-9,-19.34e-9,-15.65e-9,28.11e-9,-0.28e-9,8.12e-9,-7.52e-9,-7.55e-9,-3.69e-9], KL="KL_v20", BANK=1, /ZERNIKE, /LUCI
idl_ctrl_126283 |INF| 12648|2023-03-22 20:35:24.575018| MAIN > StartCmd
idl_ctrl_126283 |INF| 12649|2023-03-22 20:35:24.580789| MAIN > EndCmd
  • luci:.0002.fits
OptLoopDiagnostic _166752|DEB| 286|2023-03-22 20:32:21.34003 ARB-INTERFACE > Message succesfully sent
OptLoopDiagnostic_166752|DEB| 287|2023-03-22 20:32:21.34004 ARB-INTERFACE > Waiting message reply: sender=adsecarb.L@M_ADSEC, code=2024
OptLoopDiagnostic_166752|DEB| 288|2023-03-22 20:32:22.34017 ARB-INTERFACE > Error in thWaitMsg: -5001 (Timeout error)
OptLoopDiagnostic_166752|ERR| 289|2023-03-22 20:32:22.34027 MAIN > Command Save status error
OptLoopDiagnostic_166752|ERR| 290|2023-03-22 20:32:22.34032 MAIN > Error saving AdSec status: (-1) Return from search routine (item not fou
  • TN:_204712,_204731
  • 1:51
StopAO

Wed Mar 22 21:06:44.824 2023 aos.info - Updated variable AOARB.L.SL_READY: 1
Wed Mar 22 21:06:44.841 2023 aos.info - Updated variable AOARB.L.AO_READY: 0
Wed Mar 22 21:06:46.288 2023 aos.info - Updated variable AOARB.L.AO_READY: 1
Wed Mar 22 21:06:46.289 2023 aos.PresetAO{FLAO}.complete - PresetAO {FLAO}[43] - dx:nan dy:nan slNull: f1:0.0 f2:1.0 freq:1700 mag:0 nBins:1 nModes:501 r0:nan snMode:0 strehl:0 ttMod:0
Wed Mar 22 21:07:08.124 2023 aos.error - AO rotator out of al
  • rot -5
  • new preset.
Wed Mar 22 21:10:05.439 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[44] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 21:10:05.462 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 21:10:06.182 2023 aos.info - FLAO: WFS: loading setup for LUCIFER
Wed Mar 22 21:10:06.199 2023 aos.info - FLAO: WFS: configuring CCDs
Wed Mar 22 21:10:26.945 2023 aos.info - FLAO: WFS: preset done
Wed Mar 22 21:10:26.946 2023 aos.PresetAO{FLAO}.failed - PresetAO {FLAO}[44] - [AOException] Rerotator tracking out of ramge
  • rot -5
  • send preset.
  • during the preset the rotator is moving to zero
  • inside of the rpeset the rerotator went in range
  • but the preset failed.
Wed Mar 22 21:18:50.214 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[47] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 21:18:50.228 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 21:18:50.956 2023 aos.info - FLAO: WFS: loading setup for LUCIFER
Wed Mar 22 21:18:50.969 2023 aos.info - FLAO: WFS: configuring CCDs
Wed Mar 22 21:19:11.537 2023 aos.info - FLAO: WFS: preset done
Wed Mar 22 21:19:11.538 2023 aos.PresetAO{FLAO}.failed - PresetAO {FLAO}[47] - [AOException] Rerotator tracking out of ramge
  • after the w_stop-all
THRDTEST cmd: write slopecompctrl.L.OPTG.REQ r from
Warning: you must first start communication with MsgD -RTDB!

THRDTEST cmd: quit
from

THRDTEST: 6.15 64b [TH](Built: Mar 22 2023 15:19:31) - Dbg lev.:0, quiet, Line edit & history:Yes

Trying to connect to MsgD @ 127.0.0.1:9752
THRDTEST Error -67546 (Connection closed at the other end) from thStart() ..

THRDTEST cmd: write slopecompctrl.L.OPTG.REQ r from
Warning: you must first start communication with MsgD -RTDB!

THRDTEST cmd: quit

Available versions in /home/AOeng/soul:
stable
stable.20221117
stable.20230201
stable.20230202
test.20230302.offsetZ
test.20230310.ncpa-threshold
test.20230322.ncpa-threshold-and-fwFix
test.20230322.rerotator-and-fwFix
[AOeng@soul-sxwfs source]$ use_soul test.20230322.rerotator-and-fwFix
Switched live version to /home/AOeng/soul/test.20230322.rerotator-and-fwFix.
[AOeng@soul-sxwfs source]$

2:23
  • Fabio script stopped and launched before the w_start-all, the script reconects to the MSG-DM
w_start_all

Test preset
  • rot 0
Wed Mar 22 21:25:10.942 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[48] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)

  • rot -5
  • out of range
  • preset.
ed Mar 22 21:28:25.936 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[49] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 21:28:25.951 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 21:28:26.664 2023 aos.info - FLAO: WFS: loading setup for LUCIFER
Wed Mar 22 21:28:26.676 2023 aos.info - FLAO: WFS: configuring CCDs

Wed Mar 22 21:28:47.478 2023 aos.PresetAO{FLAO}.failed - PresetAO {FLAO}[49] - [AOException] presetAO: WARNING - RETRY: Rerotator tracking range error (-20005) REROT_TRACKING_ERROR

ar 22 21:28:47.478 2023 aos.PresetAO{FLAO}.failed - PresetAO {FLAO}[49] - [AOException] presetAO: WARNING - RETRY: Rerotator tracking range error (-20005) REROT_TRACKING_ERROR
Wed Mar 22 21:30:30.087 2023 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[50] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0[0],roy:0,mag:10,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
Wed Mar 22 21:30:30.102 2023 aos.info - WFS source set to: FLAOWFS
Wed Mar 22 21:30:30.793 2023 aos.info - FLAO: WFS: loading setup for LUCIFER
Wed Mar 22 21:30:30.807 2023 aos.info - FLAO: WFS: configuring CCDs
Wed Mar 22 21:30:51.752 2023 aos.info - FLAO: WFS: preset done
Wed Mar 22 21:30:51.766 2023 aos.info - Updated variable AOARB.L.RR_ENABLED: 0
Wed Mar 22 21:30:51.783 2023 aos.info - Updated variable AOARB.L.LAB_MODE: 0
Wed Mar 22 21:30:51.800 2023 aos.info - AO loop opened, Sound=leftAOOPENED.wav
Wed Mar 22 21:30:51.822 2023 aos.info - Updated variable AOARB.L.SL_READY: 1
Wed Mar 22 21:30:51.841 2023 aos.info - Updated variable AOARB.L.AO_READY: 0
Wed Mar 22 21:30:53.422 2023 aos.info - Updated variable AOARB.L.AO_READY: 1
Wed Mar 22 21:30:53.423 2023 aos.PresetAO{FLAO}.complete - PresetAO {FLAO}[50] - dx:nan dy:nan slNull: f1:0.0 f2:1.0 freq:1700 mag:0 nBins:1 nModes:501 r0:nan snMode:0 strehl:0 ttMod:0

stop

2:45

  • switch to stable.
AOeng@soul-sxwfs source]$ use_soul
Usage: use_soul [version]
Currently running: test.20230322.rerotator-and-fwFix (soul)
Available versions in /home/AOeng/soul:
stable
stable.20221117
stable.20230201
stable.20230202
test.20230302.offsetZ
test.20230310.ncpa-threshold
test.20230322.ncpa-threshold-and-fwFix
test.20230322.rerotator-and-fwFix
[AOeng@soul-sxwfs source]$
[AOeng@soul-sxwfs source]$
[AOeng@soul-sxwfs source]$ use_soul stable
Switched live version to /home/
  • power on
  • giving an error to connect to the CCD47
JoeCtrl _132698 |INF| 13|2023-03-22 21:46:58.742404| MAIN > Connecting to ts8dx:3105
JoeCtrl_132698 |ERR| 14|2023-03-22 21:48:01.741895| MAIN > Error configuring network: (-1) Return from search routine (item not found)
JoeCtrl_132698 |INF| 15|2023-03-22 21:48:02.742202| MAIN > Connecting to ts8dx:3105
JoeCtrl_132698 |ERR| 16|2023-03-22 21:48:18.797009| MAIN > Error configuring network: (-1) Return from search routine (item not found)
JoeCtrl_132698 |INF| 17|2023-03-22 21:48:19.797437| MAIN > Connecting to ts8dx:3105
JoeCtrl_132698 |ERR| 18|2023-03-22 21:48:21.797983| MAIN > Error configuring network: (-1) Return from search routine (item not found)
JoeCtrl_132698 |INF| 19|2023-03-22 21:48:22.798376| MAIN > Connecting to ts8dx:3105
JoeCtrl_132698 |ERR| 20|2023-03-22 21:48:24.798971| MAIN > Error configuring network: (-1) Return from search routine (item not found)
JoeCtrl_132698 |INF| 21|2023-03-22 21:48:25.799365| MAIN > Connecting to ts8dx:3105
JoeCtrl_132698 |INF| 22|2023-03-22 21:48:25.814993| MAIN > Network reconfigured OK
Process killed!!
JoeCtrl_132698 |FAT| 23|2023-03-22 21:56:54.359313| MAIN > ccd47.L terminated by sysprocesses.L00

  • restart the CCD47 process.
  • it shows to be connected.
  • Power on again.
  • test Luci after update the connections keys.
  • Ok

-- JuanCarlosGuerra - 22 Mar 2023
Topic revision: r3 - 24 Mar 2023, JuanCarlosGuerra
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback