Bin 3 and 4 Close loop Check.
  • LBTO team: Juan Carlos and Greg.
  • Arcetri team: Fabio.

Summary

Testing bin 3 and bin4 close loop performances. System test after a manior software update from Arcetri.

The system is setup for bin1 check and it was observed that the camera lens in Y position was at -100, the close loop check was not stable and saturation was too large.
Decided to move to check the pupil positions for bin3 and bin4. Check doen with the current pupils and there is not change, only bin1 shows the need to be shifted. It was loaded Bin1 pupil built on 20210210 and rotation check showed that the camera lens moves insode of the working range.
Also the close loop process for different binings allowed to test the updates in the software and it is shows some problems related to the changes. The system didn;t show problems related to the software changes.

The mirror had event of skip frames and open the loop..->too many skipframes.

Minor software udate.
  • -Strehl Ratio from slops is set to -1 when the GOPT is not applied.
  • -There was a bug in the removal of the notifications received but the auxloops from the message deamon. This is fixed.

Details

Startup comeup with the ocam not live.

Just a quick troubleshooting. set bin2 from HW didn't work.

The ocam2 didn't come alive.

Fabio skype call to talk about the minor soft update and he is available in the coming hours. Update the software and wfs_init.
  • Ocam2k come alive -> NOT problems.
note: it is not related to the minor update of the software. It could be that the wfs_init restarted the proceesses in a better shape. (?)

Telescope ready by 11.
Shutter position and WFS mag.
  • Shutter =49 WFS mag=15.3
  • Shutter=44 mag=16.9
  • shutter=58,mag=13.2
  • shutter=51 mag=14.6
Deploy argos. 17:58.

SetShell

WfsArbitrator _62014|INF| 46|2021-02-23 18:12:51.189999| WFS > Python command: WfsArbScripts.presetAO(self, sensor = 'FLAOWFS', instr = 'LUCIFER', mode = 'ACE-AO', starxpos = -0, starypos = 0, telElevation = 0, derotAngle = 0, starMag = 8.5, starColor = 0, r0 = 0, badseeing = 0, v0 = 0)

PSF on the CCD47.
Acquire Ref. It didn;t close the loop properly.
WFS Fastdiag down.
Restarted and try again.

WfsArbitrator _143614|INF| 40|2021-02-23 18:26:50.432547| WFS > Python command: WfsArbScripts.presetAO(self, sensor = 'FLAOWFS', instr = 'LUCIFER', mode = 'ACE-AO', starxpos = -0, starypos = 0, telElevation = 0, derotAngle = 0, starMag = 8.5, starColor = 0, r0 = 0, badseeing = 0, v0 = 0)

Center pupil failed. Secondary in fault.
18:30 -Power OFF sec.

WfsArbitrator _143614|ERR| 79|2021-02-23 18:30:30.404490| MAIN > Cannot read telescope rotator position: [AOVarException] thGetVar error (code -5001) Timeout errorVariable: AOS.L.ROTATOR.ANGLE@M_ADSEC (type REAL_VARIABLE, len 1)

WfsArbitrator _143614|ERR| 79|2021-02-23 18:30:30.404490| MAIN > .

WfsArbitrator _143614|ERR| 80|2021-02-23 18:30:31.404752| MAIN > Cannot read the telescope elevation: [AOVarException] thGetVar error (code -5001) Timeout errorVariable: AOS.TEL.EL@M_ADSEC (type REAL_VARIABLE, len 1)

WfsArbitrator _143614|ERR| 80|2021-02-23 18:30:31.404752| MAIN > .

WfsArbitrator _143614|ERR| 81|2021-02-23 18:30:33.405187| MAIN > Cannot read telescope rotator position: [AOVarException] thGetVar error (code -5001) Timeout errorVariable: AOS.L.ROTATOR.ANGLE@M_ADSEC (type REAL_VARIABLE, len 1)

WfsArbitrator _143614|ERR| 81|2021-02-23 18:30:33.405187| MAIN > .

WfsArbitrator _143614|ERR| 82|2021-02-23 18:30:34.405424| MAIN > Cannot read the telescope elevation: [AOVarException] thGetVar error (code -5001) Timeout errorVariable: AOS.TEL.EL@M_ADSEC (type REAL_VARIABLE, len 1)

WfsArbitrator _143614|ERR| 82|2021-02-23 18:30:34.405424| MAIN > .

WfsArbitrator _143614|ERR| 83|2021-02-23 18:30:36.405820| MAIN > Cannot read telescope rotator position: [AOVarException] thGetVar error (code -5001) Timeout errorVariable: AOS.L.ROTATOR.ANGLE@M_ADSEC (type REAL_VARIABLE, len 1)

WfsArbitrator _143614|ERR| 83|2021-02-23 18:30:36.405820| MAIN > .

WfsArbitrator _143614|ERR| 84|2021-02-23 18:30:37.406071| MAIN > Cannot read the telescope elevation: [AOVarException] thGetVar error (code -5001) Timeout errorVariable: AOS.TEL.EL@M_ADSEC (type REAL_VARIABLE, len 1)

WfsArbitrator _143614|ERR| 84|2021-02-23 18:30:37.406071| MAIN > .

WfsArbitrator _143614|ERR| 85|2021-02-23 18:30:38.406783| MAIN > Cannot read telescope rotator position: [AOVarException] thGetVar error (code -3006) Variable not found errorVariable: AOS.L.ROTATOR.ANGLE@M_ADSEC (type REAL_VARIABLE, len 1)

Processes disconnected from Msgdeamon.
18:46 Called Fabio. Checking.

it lookis like that after the adsec restart the WFS wasn't restarted.It is important to align the two systems.
After power off Adsec it requires a restart of the WFS.
wfs_stop_all/wfs_start_all.

Restarted auxloop for Fabio.
bin 1 pupil file: 20190909_203854.

WfsArbitrator _196332|INF| 239|2021-02-23 19:05:12.983814| WFS > Executing presetAO...

WfsArbitrator _196332|INF| 240|2021-02-23 19:05:12.983939| WFS > Python command: WfsArbScripts.presetAO(self, sensor = 'FLAOWFS', instr = 'LUCIFER', mode = 'ACE-AO', starxpos = -0, starypos = 0, telElevation = 0, derotAngle = 0, starMag = 8.5, starColor = 0, r0 = 0, badseeing = 0, v0 = 0)

MOVE to BIN3 PUPIL WOBBLE ROT.

873Hz. Argoscalunit shutter: 51 WFS counts:mag 15.

WfsArbitrator _196332|INF| 253|2021-02-23 19:29:15.375490| WFS > Executing presetAO...
WfsArbitrator_196332|INF| 254|2021-02-23 19:29:15.375606| WFS > Python command: WfsArbScripts.presetAO(self, sensor = 'FLAOWFS', instr = 'LUCIFER', mode = 'ACE-AO', starxpos = -0, starypos = 0, telElevation = 0, derotAngle = 0, starMag = 15, starColor = 0, r0 = 0, badseeing = 0, v0 = 0)
  • Disturbance s2.3
  • Optimize gain ->Ok.
Gopt->Ok
bin3_gopt_feb23.png


bin3_feb23_rot.png

bin3_feb23_rot_puperr.png

BIN3_UP_FEB23.png

bin3_puperr_up_deb23.png

BIN 4

ArgeosCalUnit:44 freq:268Hz WFS:

WfsArbitrator _196332|INF| 878|2021-02-23 20:13:15.274704| WFS > Executing presetAO...

WfsArbitrator _196332|INF| 879|2021-02-23 20:13:15.274833| WFS > Python command: WfsArbScripts.presetAO(self, sensor = 'FLAOWFS', instr = 'LUCIFER', mode = 'ACE-AO', starxpos = -0, starypos = 0, telElevation = 0, derotAngle = 0, starMag = 16.6, starColor = 0, r0 = 0, badseeing = 0, v0 = 0)

OPTIMIZE GAIN->OK

bin4_s2.png

CHANGE DISTURBACE TO S1.

Optimaze gain->OK

bin4_s2.png

WfsArbitrator _196332|INF| 1178|2021-02-23 20:35:08.094883| OPTLOOPDIAG > Executing save...

WfsArbitrator _196332|INF| 1179|2021-02-23 20:35:10.004224| WFS > Executing saveStatus...

WfsArbitrator _196332|INF| 1180|2021-02-23 20:35:10.004316| WFS > Python command: WfsArbScripts.saveStatus(self, sensor = 'FLAOWFS', instr = 'LUCIFER', outputFile = '/local/aomeas/adsec_data/20210223/Data_20210223_203508/wfs.fits')

WfsArbitrator _196332|INF| 1181|2021-02-23 20:35:10.033890| WFS > Command returns success .
WfsArbitrator_196332|INF| 1182|2021-02-23 20:35:12.525034| MAIN > Camera lens tracking: AdSec is skipping frames, camera lens not moving
WfsArbitrator_196332|INF| 1183|2021-02-23 20:35:16.423747| MAIN > Camera lens tracking: AdSec is skipping frames, camera lens not moving

WfsArbitrator _196332|INF| 1184|2021-02-23 20:35:20.571452| MAIN > Camera lens tracking: AdSec is skipping frames, camera lens not moving
WfsArbitrator_196332|INF| 1185|2021-02-23 20:35:24.742862| MAIN > Camera lens tracking: AdSec is skipping frames, camera lens not movin

Adsec log:

fastdiagn_129565 |INF| 119647|2021-02-23 20:35:01.332471| MAIN > Calling aos_offload [FastDiagnostic.cpp:1197]

fastdiagn_129565 |WAR| 119648|2021-02-23 20:35:02.312598| FUNCTWARNING > FunctWarning CHFFPURECURRENT-0003 -0.654376

fastdiagn_129565 |WAR| 119649|2021-02-23 20:35:02.316412| FUNCTWARNING > FunctWarning CHFFPURECURRENT-0003 -0.665871

fastdiagn_129565 |WAR| 119650|2021-02-23 20:35:02.320161| FUNCTWARNING > FunctWarning CHFFPURECURRENT-0003 -0.666503

Preset again

WfsArbitrator _196332|INF| 1244|2021-02-23 20:37:39.438612| WFS > Python command: WfsArbScripts.presetAO(self, sensor = 'FLAOWFS', instr = 'LUCIFER', mode = 'ACE-AO', starxpos = -0, starypos = 0, telElevation = 0, derotAngle = 0, starMag = 16.6, starColor = 0, r0 = 0, badseeing = 0, v0 = 0)
  • Opt gain->OK
  • Gopt->Ok
  • bin4_second_set_gopt.png

loop open skip frames

idl_129502 |INF| 1246|2021-02-23 20:54:40.492534| MAIN > Forces out of safe range ID: #111 #138 #166

idl_129502 |INF| 1247|2021-02-23 20:54:40.492566| MAIN > SafeSkipFrames caused by forces out of range detected.

idl_129502 |INF| 1248|2021-02-23 20:54:40.492593| MAIN > Recovery from any SafeSkipFrames started...

fastdiagn_129565 |INF| 125232|2021-02-23 20:54:36.536914| MAIN > Received HighOrderOffloadMatrix change /home/aoeng/soul/aoroot/calib/adsec/672b/data/null_pmhofor.fits [FastDiagnostic.cpp:294]

fastdiagn_129565 |INF| 125233|2021-02-23 20:54:36.537007| MAIN > Changed reference positions [FastDiagnostic.cpp:260]

fastdiagn_129565 |INF| 125234|2021-02-23 20:54:36.537089| MAIN > Changed reference forces: Averaged reference force is -0.0407116 [FastDiagnostic.cpp:281]

fastdiagn_129565 |INF| 125235|2021-02-23 20:54:36.541010| MAIN > Received HighOrderOffloadMatrix change /home/aoeng/soul/aoroot/calib/adsec/672b/data/null_pmhofor.fits [FastDiagnostic.cpp:294]

fastdiagn_129565 |INF| 125236|2021-02-23 20:54:36.541206| MAIN > Changed reference positions [FastDiagnostic.cpp:260]

fastdiagn_129565 |INF| 125237|2021-02-23 20:54:36.541320| MAIN > Changed reference forces: Averaged reference force is -0.0407116 [FastDiagnostic.cpp:281]

fastdiagn_129565 |INF| 125238|2021-02-23 20:54:37.051559| FAST > Processed 234 cycles of 7702 vars in 0.97 s ( 241 Hz). HW frame rate 761 [DiagnApp.cpp:832]

fastdiagn_129565 |INF| 125239|2021-02-23 20:54:37.051748| MAIN > Calling aos_offload [FastDiagnostic.cpp:1197]

fastdiagn_129565 |WAR| 125240|2021-02-23 20:54:37.176014| FUNCTWARNING > FunctWarning CHFFPURECURRENT-0442 -0.62044

fastdiagn_129565 |WAR| 125241|2021-02-23 20:54:37.180005| FUNCTWARNING > FunctWarning CHFFPURECURRENT-0442 -0.625537

fastdiagn_129565 |WAR| 125242|2021-02-23 20:54:37.184053| FUNCTWARNING > FunctWarning CHFFPURECURRENT-0442 -0.61955

fastdiagn_129565 |WAR| 125243|2021-02-23 20:54:37.188231| FUNCTWARNING > FunctWarning CHFFPURECURRENT-0442 -0.627498

fastdiagn_129565 |WAR| 125244|2021-02-23 20:54:37.192404| FUNCTWARNING > FunctWarning CHFFPURECURRENT-0442 -0.625399

fastdiagn_129565 |WAR| 125245|2021-02-23 20:54:37.196415| FUNCTWARNING > FunctWarning CHFFPURECURRENT-0442 -0.611056

Recover. Close loop and optimize gain.

optmize gain->ok

Gopt->ok

bin4_third_set_gopt.png

bin4_up_down_feb23.png

BIN1

ARGOSCALUNIT: FULL OPEN.

WfsArbitrator _196332|INF| 1894|2021-02-23 21:24:22.840221| WFS > Python command: WfsArbScripts.presetAO(self, sensor = 'FLAOWFS', instr = 'LUCIFER', mode = 'ACE-AO', starxpos = -0, starypos = 0, telElevation = 0, derotAngle = 0, starMag = 10, starColor = 0, r0 = 0, badseeing = 0, v0 = 0)

Optimize gain->failed.

WfsArbitrator _196332|INF| 1996|2021-02-23 21:28:18.244803| MAIN > Camera lens tracking: AdSec is skipping frames, camera lens not moving
WfsArbitrator_196332|ERR| 1997|2021-02-23 21:28:22.455110| MAIN > OptimizeGain: FATAL - Safe skip active - cannot proceed (-10045) IDL_SAFE_SKIP_ACTIVE
WfsArbitrator_196332|INF| 1998|2021-02-23 21:28:23.312231| COMMANDHANDLER > FSM (status: AOSet) has received command 2023 (EnableDisturb)
WfsArbitrator_196332|INF| 1999|2021-02-23 21:28:23.312311| WFS > Executing EnableDisturb..
  • Second optimize gain->OK
  • Gopt->OK
Skip frame. Open the loop. Camera lens in Y direction saturated (-100)
The error is too large in Y direction. After the loop open by skip frames, it was decided to load the new pupils and tested.

Apply a default flat.
shape of the Adsec when skipping in a bad position of the camera lens. It build up a high order modes...
mirrorshape_feb23.png

Load the pupils 20210210_205151.
Offset applied to file.
PresetAO.

WfsArbitrator _196332|INF| 2036|2021-02-23 21:28:51.972367| WFS > Python command: WfsArbScripts.presetAO(self, sensor = 'FLAOWFS', instr = 'LUCIFER', mode = 'ACE-AO', starxpos = -0, starypos = 0, telElevation = 0, derotAngle = 0, starMag = 10, starColor = 0, r0 = 0, badseeing = 0, v0 = 0)
  • optgain->ok
  • gopt->ok. After a second try.

bin1_optgain_feb23.png

at rot 90 there is saturtation in positive direction Y (+100).
pupil was not behaving correctly (stack in that position--??). restart and it seems going better.

bin1_pupshift20210210.png

bin1_up_down_pup20210210.png

pupil 1after the shift looks better.....

??? rot 40 ->posY to -100

22:33 STOP

  • warmup ocam.
  • wfs_idle

-- JuanCarlosGuerra - 23 Feb 2021
I Attachment Action Size Date Who Comment
BIN3_UP_FEB23.pngpng BIN3_UP_FEB23.png manage 260 K 23 Feb 2021 - 20:18 JuanCarlosGuerra  
bin1_gopt_feb23.pngpng bin1_gopt_feb23.png manage 449 K 23 Feb 2021 - 21:37 JuanCarlosGuerra bin1_optgain
bin1_optgain_feb23.pngpng bin1_optgain_feb23.png manage 340 K 23 Feb 2021 - 21:55 JuanCarlosGuerra  
bin1_pupil_feb23.pngpng bin1_pupil_feb23.png manage 244 K 23 Feb 2021 - 21:37 JuanCarlosGuerra bin1_pup
bin1_pupildefault_puperr_feb23.pngpng bin1_pupildefault_puperr_feb23.png manage 415 K 23 Feb 2021 - 21:52 JuanCarlosGuerra  
bin1_pupshift20210210.pngpng bin1_pupshift20210210.png manage 285 K 23 Feb 2021 - 21:59 JuanCarlosGuerra  
bin1_up_down_pup20210210.pngpng bin1_up_down_pup20210210.png manage 321 K 23 Feb 2021 - 22:29 JuanCarlosGuerra  
bin3_feb23.pngpng bin3_feb23.png manage 203 K 23 Feb 2021 - 20:13 JuanCarlosGuerra  
bin3_feb23_rot.pngpng bin3_feb23_rot.png manage 354 K 23 Feb 2021 - 20:02 JuanCarlosGuerra bin3_rot
bin3_feb23_rot_puperr.pngpng bin3_feb23_rot_puperr.png manage 223 K 23 Feb 2021 - 20:18 JuanCarlosGuerra  
bin3_gopt_feb23.pngpng bin3_gopt_feb23.png manage 263 K 23 Feb 2021 - 20:09 JuanCarlosGuerra Gopt_bin3
bin3_puperr_up_deb23.pngpng bin3_puperr_up_deb23.png manage 350 K 25 Feb 2021 - 08:18 JuanCarlosGuerra  
bin4_s2.pngpng bin4_s2.png manage 297 K 23 Feb 2021 - 20:21 JuanCarlosGuerra Gopt_bin4_s2.3
bin4_second_set_gopt.pngpng bin4_second_set_gopt.png manage 318 K 23 Feb 2021 - 20:48 JuanCarlosGuerra  
bin4_third_set_gopt.pngpng bin4_third_set_gopt.png manage 351 K 23 Feb 2021 - 21:25 JuanCarlosGuerra  
bin4_up_down_feb23.pngpng bin4_up_down_feb23.png manage 220 K 23 Feb 2021 - 21:25 JuanCarlosGuerra bin4_up_down
bin_pupildefault_feb23.pngpng bin_pupildefault_feb23.png manage 244 K 23 Feb 2021 - 21:52 JuanCarlosGuerra  
mirrorshape_feb23.pngpng mirrorshape_feb23.png manage 319 K 23 Feb 2021 - 22:49 JuanCarlosGuerra mirror_shape
Topic revision: r3 - 25 Feb 2021, JuanCarlosGuerra
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback