Details.

timeout bcu2k

or_185590|INF| 79|2022-03-16 00:19:42.697926| MAIN > Ocam2K Setup Failed, Powering it Off
WfsArbitrator_185590|INF| 80|2022-03-16 00:19:42.698047| MAIN > Timeout waiting for ocam2.L.STATUS - TIMEOUT_ERROR
WfsArbitrator_185590|ERR| 81|2022-03-16 00:19:42.698097| MAIN > Exception: Timeout waiting for ocam2.L.STATUS
WfsArbitrator_185590|ERR| 82|2022-03-16 00:19:42.698142| MAIN > Timeout waiting for ocam2.L.STATUS (-5001) TIMEOUT_ERROR
WfsArbitrator_185590|DEB| 83|2022-03-16 00:22:07.359437| MAIN > arbFunc TurnSystemOn (, 'FLAOWFS', '', 'WFS_complete_(wit

2nd try.

go through the logs to catch this type of events.

ocam not live.
  • restart bcu2k
  • no luck
  • restart masterdiagnostic
  • ok.
move camera lens to a median positions, eg; -8.10,12.2.

The first movement applied seems that the axis went to negative positions.The slider starts at -100 for both axis.

camera_lens_move20220315.png

saving act positions at 30 degrees.

TN:20220315_013720

data not saved.

saving ocam fames.

_014027

wind 6m/s az 220

EL 64 for OSCO

Wed Mar 16 01:56:31.117 2022 aos.Stop{FLAO}.complete - Stop{FLAO}[1831]
Wed Mar 16 01:56:37.070 2022 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[1832] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:-2.99182[-2.99182],soy:2.99182,rox:-2.9334[-2.9334],roy:3.12336,mag:7.52,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)
  • camera lens goes to 0,0
  • ncpa off.
  • SR ~68% in FeII
  • gopt:
  • 1.38,0.34,0.32
  • good PSF.
AOS GUI stop AO the FLAO gui didn;t show the CANCEL PRESET BUTTON available.

oscillation of the loop, skipping frame with the NCPA ON

ncap_on_off_20220316.png

Wed Mar 16 02:28:17.431 2022 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[1847] (aomode:TTM-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:-0.0119673[-0.0119673],soy:0.568446,rox:-44.2388[-44.2388],roy:-10.1221,mag:13.94,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)

bin 4, 400 Hz.
  • mag estimated 15.5
7:30 dimm 1"

slope rms 0.5

tt jitter look corrcting.

7:38 dimm 1.26"

7:44 dimm 1.5"

tn:_025241

_TT 0.5 _025441+ccd47

_TT 0.6 025606

rms 0.55

TT1.05,0.45,0.05

_033115

crossing meriadian.

loop open (?)

fsm_stop_ao sent (?)

Start not visible in the CCD47

Do a new preset.

EL 87

8:59 dimm 1"

rrms 0.55

0.65,0.35,0.05

gain TT to 0.8 oscillating.the image of the pupil jumping diagonal left top to right botton

_032929

Check HO Offload request..(?)

Wed Mar 16 02:28:17.431 2022 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[1847] (aomode:TTM-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:-0.0119673[-0.0119673],soy:0.568446,rox:-44.2388[-44.2388],roy:-10.1221,mag:13.94,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)

03:41:05.985400| MAIN > Request: Pause()
python_55689 |DEB| 515|2022-03-16 03:41:06.284763| MAIN > Status after command: LoopSuspended
python_55689 |INF| 516|2022-03-16 03:41:09.193454| MAIN > Request: OffsetXY ((1.0959328413009644, 0.2797677218914032))
python_55689 |INF| 517|2022-03-16 03:41:10.120588| MAIN > ... 1 equal lines suppressed
python_55689 |ERR| 518|2022-03-16 03:41:10.120677| MAIN > Alert from AdSec [177] - SkipFrame
python_55689 |DEB| 519|2022-03-16 03:41:10.120720| MAIN > Sending to 3 clients: SkipFrame
python_55689 |DEB| 520|2022-03-16 03:41:13.505398| MAIN > Status after command: LoopSus

03:41:13.505441| MAIN > Command interrupted

Wed Mar 16 04:21:16.521 2022 aos.PresetAO{FLAO}.started - PresetAO {FLAO}[2009] (aomode:ACE-AO,focst:bentGregorianFront,instr:LUCIFER,wfs:FLAOWFS,sox:0[0],soy:0,rox:0.207278[0.207278],roy:0.615316,mag:14,cindex:0,,elev:nan,rotang:nan,grvang:nan,r0:0,skybr:nan,windsp:nan,winddir:nan)

cloosing core

9:34 dimm 1.54"

optimize +gopt

optimize->6,1.3,0.01

offset to sky first

ator_185590|DEB| 1895|2022-03-16 04:34:37.954977| MAIN > arbFunc offsetXYinternal (, 'FLAOWFS', 'LUCIFER', -54.274299999999997, -443.32499999999999, True)
WfsArbitrator_185590|INF| 1896|2022-03-16 04:34:37.955214| MAIN > offsetXY(): starting offset of -52.107, -443.585 mm
WfsArbitrator_185590|INF| 1897|2022-03-16 04:34:37.955979| MAIN > offsetXYZ(): Focus correction (absolute): 0.000 mm
WfsArbitrator_185590|ERR| 1898|2022-03-16 04:34:37.956450| MAIN > RETRY: Offset outside FoV limits. Target xyz is -129.62 -494.67 52.90 (-5002) VALUE_OUT_OF_RANGE_

TN:_045028

9:55 dimm 1.25"

10:05 dimm 1.6"

_050843

Wed Mar 16 05:25:23.690 2022 aos.AcquireRef{FLAO}.complete - AcquireRef {FLAO}[2085] - dx:0 dy:0 slNull: f1:0.0 f2:1.0 freq:739 mag:0 nBins:3 nModes:90 r0:nan snMode:0 strehl:0 ttMod:0

optimize didn;t go for the first pass.

moving gain manually, looking at the first two plots.

3.64,1.10,0.1

The core is a bit more defuse than the previous one.

4.0, 1.10,0.15

_053434

10:36 dimm 1.1"

_054937

_055456

saw a seeing valur of 1.5" maybe larger, the image in the CCD47 was almost desapair.

Request: OffsetXY ((-0.45711889863014221, 0.66912847757339478))
python_55689 |DEB| 1078|2022-03-16 05:54:29.317838| MAIN > Status after command: LoopSuspended
python_55689 |INF| 1079|2022-03-16 05:54:29.865199| MAIN > Request: Resume()
python_55689 |DEB| 1080|2022-03-16 05:54:29.992357| MAIN > Status after command: LoopClosed
python_55689 |INF| 1081|2022-03-16 05:55:39.448801| MAIN > Loop fault detected. Opening loop.
python_55689 |INF| 1082|2022-03-16 05:55:44.782875| MAIN > ... 1 equal lines suppressed
python_55689 |ERR| 1083|2022-03-16 05:55:44.782928| MAIN > Alert from AdSec [177] - SkipFrame
python_55689 |DEB| 1084|2022-03-16 05:55:44.782968| MAIN > Sending to 3 clients: SkipFrame
python_55689 |INF| 1085|2022-03-16 05:55:50.213316| MAIN > Request: Pause()
python_55689 |DEB| 1086|2022-03-16 05:55:50.213428| MAIN > Status after command: LoopFault

reclose lowering the TT gain to 3.5,1.15,0.2

Screen Shot 2022-03-16 at 12.06.28 AM.png

new target

too faint for the axcq in CCD47, bin3 setup

requested to go to 16.5.

hard to see the object in the CCD47.

Failed to acquired.

abort this target.

new trget.

core concentrated.

optimize gain failed to get the second pass.

Set gain manually->3.6,1.20,0.10

luci sent again the preset after skipping one slot....

diff = img2-img1

optimize

found a tT 1.4

gopt applied.

_064603

_064842

11:49 dimm 1.17

12:11 dimm 0.8"

_071121

Wed Mar 16 07:19:32.307 2022 aos.PresetAO{FLAO}.complete - PresetAO {FLAO}[2245] - dx:nan dy:nan slNull: f1:0.0 f2:1.0 freq:839 mag:0 nBins:3 nModes:90 r0:nan snMode:0 strehl:0 ttMod:0

optimize 6 1.3,0.1

gopt ok

0.66,0.16,0.02

optimize_12p26.png

12:27 dimm 0.8"

resume offset come just one arcsec off and the system pull in with skip frames and change AOSet.

Looking at the hotspot cros in the CCD47 arrvied to the top right

07:25:53.168711| MAIN > Request: OffsetXY ((6.154700756072998, -36.21356201171875))
python_55689 |DEB| 1580|2022-03-16 07:25:53.171411| MAIN > Status after command: LoopSuspended
python_55689 |ERR| 1581|2022-03-16 07:25:53.171433| MAIN > [AOException] RETRY: Offset outside FoV limits. Target xyz is -77.05 -107.68 52.90 (-5002) VALUE_OUT_OF_RANGE_ERROR
python_55689 |INF| 1582|2022-03-16 07:28:02.991222| MAIN > Request: Pause()
python_55689 |DEB| 1583|2022-03-16 07:28:02.991291| MAIN > Status after command: LoopSuspended
python_55689 |INF| 1584|2022-03-16 07:28:05.940246| MAIN > Request: OffsetXY ((0.16224236786365509, -0.38852313160896301))
python_55689 |DEB| 1585|2022-03-16 07:28:09.350698| MAIN > Status after command: LoopSuspended
python_55689 |INF| 1586|2022-03-16 07:28:16.057579| MAIN > Request: Resume()
python_55689 |DEB| 1587|2022-03-16 07:28:16.221229| MAIN > Status after command: LoopClosed
python_55689 |INF| 1588|2022-03-16 07:28:18.300358| MAIN > Loop fault detected. Opening loop.
python_55689 |INF| 1589|2022-03-16 07:28:22.903218| MAIN > ... 1 equal lines suppressed
python_55689 |ERR| 1590|2022-03-16 07:28:22.903259| MAIN > Alert from AdSec [177] - SkipFrame
python_55689 |DEB| 1591|2022-03-16 07:28:22.903285| MAIN > Sending to 3 clients: SkipFrame

reclose.

07:29:37.561362| MAIN > Status after command: LoopSuspended
python_55689 |INF| 1599|2022-03-16 07:29:40.287870| MAIN > Request: OffsetXY ((-0.019858594983816147, 0.73612064123153687))
python_55689 |DEB| 1600|2022-03-16 07:29:42.395913| MAIN > Status after command: LoopSuspended
python_55689 |INF| 1601|2022-03-16 07:29:47.516039| MAIN > Request: Resume()
python_55689 |DEB| 1602|2022-03-16 07:29:47.693473| MAIN > Status after command: LoopClosed
python_55689 |ERR| 1603|2022-03-16 07:29:47.743611| MAIN > Alert from AdSec [0] - Error
python_55689 |DEB| 1604|2022-03-16 07:29:47.743647| MAIN > Sending to 3 clients: Error

Reclose loop.

The reclose has to be carefully because it is off and the first commands pull in the star and it is observed skip frames.

EL 70

12:34 adsc_stop/start

pupil check didn;t get any values and it didn;t move the camera lens.

Cancel preset.

sed again.

4.7,1.5,0.1

ccd47_acq_12p51.png

reclose. applying gain the loop wne tinto pause.

adsec failure.

new preset.

Ok.

Loop open.

Reclose apply gain and during the gain applying load gain failed and adsec in Failure state.

Send preset again.

gopt->0.35,0.17,0.02

offset sky

offset back->skip frame open loop.

08:13:09.579920| MAIN > Request: Pause()
python_68173 |DEB| 97|2022-03-16 08:13:09.580001| MAIN > Status after command: LoopSuspended
python_68173 |INF| 98|2022-03-16 08:13:12.505602| MAIN > Request: OffsetXY ((0.16178765892982483, -0.38729727268218994))
python_68173 |DEB| 99|2022-03-16 08:13:15.815661| MAIN > Status after command: LoopSuspended
python_68173 |INF| 100|2022-03-16 08:13:22.170797| MAIN > Request: Resume()
python_68173 |DEB| 101|2022-03-16 08:13:22.332802| MAIN > Status after command: LoopClosed
python_68173 |INF| 102|2022-03-16 08:13:25.011766| MAIN > Loop fault detected. Opening loop.
python_68173 |ERR| 103|2022-03-16 08:13:29.803896| MAIN > Alert from AdSec [177] - SkipFrame
python_68173 |DEB| 104|2022-03-16 08:13:29.803941| MAIN > Sending to 3 clients: SkipFrame
python_68173 |INF| 105|2022-03-16 08:13:44.289584| MAIN > Request: ReCloseLoop ()

08:29:09.546382| MAIN > Request: OffsetXY ((6.430851936340332, -35.471427917480469))
python_68173 |DEB| 173|2022-03-16 08:29:09.548782| MAIN > Status after command: LoopSuspended
python_68173 |ERR| 174|2022-03-16 08:29:09.548809| MAIN > [AOException] RET

offsetXY -5.2, -35.86

good one...

0:40.534764| MAIN > offsetXY(): starting offset of -5.481, -36.516 mm
WfsArbitrator_185590|INF| 4218|2022-03-16 08:40:40.535716| MAIN > offsetXYZ(): Focus correction (absolute): 0.000 mm
WfsArbitrator_185590|ERR| 4219|2022-03-16 08:40:40.536098| MAIN > RETRY: Offset outside FoV limits. Target xyz is -76.54 -108.11 52.90 (-5002) VALUE_OUT_OF_RANGE_ERROR
WfsArbitrator_185590|DEB| 4220|2022-03-16 08:42:52.077717| MAIN > arbFunc offsetXYinternal (, 'FLAOWFS', 'LUCIFER', 0.95542800000000006, -0.549647, True)
WfsArbitrator_185590|INF| 4221|2022-03-16 08:42:52.078026| MAIN > offsetXY(): starting offset of 0.958, -0.545 mm
WfsArbitrator_185590|INF| 4222|2022-03-16 08:42:52.078836| MAIN > offsetXYZ(): Focus correction (absolute): 0.000 mm

1:46 clouds passing

good offset.

08:46:00.364695| MAIN > offsetXY(): starting offset of -6.504, -36.575 mm

_085039

modes SR~0.13

08:51:17.819819| MAIN > offsetXY(): starting offset of -5.738, -36.553 mm

good.

offset back is running loop in pause and resume , the star moves to the right positions.

_090101

09:04:17.935948| MAIN > Request: OffsetXY ((-0.56042963266372681, 0.23445023596286774))
python_68173 |DEB| 306|2022-03-16 09:04:20.544960| MAIN > Status after command: LoopSuspended
python_68173 |INF| 307|2022-03-16 09:04:29.061496| MAIN > Request: Resume()
python_68173 |DEB| 308|2022-03-16 09:04:29.224489| MAIN > Status after command: LoopClosed
python_68173 |INF| 309|2022-03-16 09:04:33.251141| MAIN > Loop fault detected. Opening loop.
python_68173 |INF| 310|2022-03-16 09:04:39.603803| MAIN > ... 1 equal lines suppressed
python_68173 |ERR| 311|2022-03-16 09:04:39.603849| MAIN > Alert from AdSec [177] - SkipFrame
python_68173 |DEB| 312|2022-03-16 09:04:39.603879| MAIN > Sending to 3 clients: SkipFrame

New target.

Wed Mar 16 09:07:02.325 2022 aos.PresetAO{FLAO}.complete - PresetAO {FLAO}[2355] - dx:nan dy:nan slNull: f1:0.0 f2:1.0 freq:750 mag:0 nBins:3 nModes:90 r0:nan snMode:0 strehl:0 ttMod:0

2:14 dimm 1.2"

optimize_2p20.png

gopt->0.16,0.07,0.01

2:20 dimm 1.6"

tn:_092225

09:28:46.511651| MAIN > Status after command: LoopSuspended
python_68173 |INF| 445|2022-03-16 09:28:50.031685| MAIN > Request: OffsetXY ((-0.27104300260543823, 0.53833800554275513))
python_68173 |DEB| 446|2022-03-16 09:28:53.341031| MAIN > Status after command: LoopSuspended
python_68173 |INF| 447|2022-03-16 09:28:58.510281| MAIN > Request: Resume()

after this one the PSF is off the ccd47 hotspot.

from AOS pause. move the star back to the hotspot and resume, the resume showed that the star was driffing away.

another pause/resume that didn';t succeded.resume, the loop opens.

Repoint the telescope to the Py and reclose and apply gain.

looks like stabel and proceed with Luci.

2:41 dimm 1.25"

_094343

09:43:21.550757| MAIN > Request: OffsetXY ((-1.0549657344818115, -0.80195659399032593))
python_68173 |DEB| 491|2022-03-16 09:43:24.460470| MAIN > Status after command: LoopSuspended
python_68173 |INF| 492|2022-03-16 09:43:32.250849| MAIN > Request: Resume()
python_68173 |DEB| 493|2022-03-16 09:43:32.380292| MAIN > Status after command: LoopClosed
python_68173 |INF| 494|2022-03-16 09:44:29.043662| MAIN > Loop fault detected. Opening loop.
python_68173 |INF| 495|2022-03-16 09:44:34.614278| MAIN > ... 1 equal lines suppressed
python_68173 |ERR| 496|2022-03-16 09:44:34.614346| MAIN > Alert from AdSec [177] - SkipFrame
python_68173 |DEB| 497|2022-03-16 09:44:34.614391| MAIN > Sending to 3 clients: SkipFrame
python_68173 |INF| 498|2022-03-16 09:44:52.442253| MAIN > Request: Pause()

loop open after the resume.

repoint/reclose and apply gain

2:48 clouds passing.

3:08 ESM

10:07:38.471343| MAIN > Request: PresetAO (
python_68173 |INF| 590|2022-03-16 10:07:38.471343| MAIN > . {
python_68173 |INF| 590|2022-03-16 10:07:38.471343| MAIN > . mode = FLAOTT

rms->0.57

optimize.

optimize_esm_3p20.png

opt->1.08,1.06,0.10

jitter_esm_03p20.png

clouds passing

3:23 dimm 1.7"

loop opne

10:25:50.279449| MAIN > Request: OffsetXY ((0.18244622647762299, 21.917257308959961))
python_68173 |WAR| 697|2022-03-16 10:26:09.512913| MAIN > Communication with FLAOWFS lost
python_68173 |INF| 698|2022-03-16 10:26:10.733735| MAIN > Communication with FLAOWFS established
python_68173 |DEB| 699|2022-03-16 10:26:10.815857| MAIN > Status after command: LoopSuspended
python_68173 |INF| 700|2022-03-16 10:26:11.538249| MAIN > Request: Resume()
python_68173 |DEB| 701|2022-03-16 10:26:11.756097| MAIN > Status after command: LoopClosed
python_68173 |INF| 702|2022-03-16 10:26:48.945045| MAIN > Loop fault detected. Opening loop.
python_68173 |INF| 703|2022-03-16 10:26:58.173958| MAIN > ... 1 equal lines suppressed
python_68173 |ERR| 704|2022-03-16 10:26:58.174046| MAIN > Alert from AdSec [177] - SkipFrame
python_68173 |DEB| 705|2022-03-16 10:26:58.174095| MAIN > Sending to 3 clients: SkipFrame

10:26:11.746964| COMMANDHANDLER > FSM (status: AORunning) has received command 2029 (TTOffload)
AdSecArbitrator_68144|INF| 1746|2022-03-16 10:26:11.755682| COMMANDHANDLER > Command TTOffload (code 2029) successfully completed
AdSecArbitrator_68144|WAR| 1747|2022-03-16 10:26:47.938911| MAIN > Skip frame detected!
AdSecArbitrator_68144|WAR| 1748|2022-03-16 10:26:48.944679| MAIN > Skip frame detected!
AdSecArbitrator_68144|INF| 1749|2022-03-16 10:26:52.794381| MAIN > Received cmd: StopAo (1)
AdSecArbitrator_68144|INF| 1750|2022-03-16 10:26:52.794479| COMMANDHANDLER > FSM (status: AORunning) has received command 2015 (Stop Ao)

reclose.

1.5,0.5,0.05

3:32 dimm 2.3"

3:33 mag 16.7

loop open

gcs_3p39.png

ccd47_3p30.png

The signal in the Py is almost sky......

3:38 STOP AO

clouds incresing

seeing getting bad.

the rest of the night the perspective looks bad.

-- JuanCarlosGuerra - 16 Mar 2022
I Attachment Action Size Date Who Comment
Screen Shot 2022-03-16 at 12.06.28 AM.pngpng Screen Shot 2022-03-16 at 12.06.28 AM.png manage 299 K 16 Mar 2022 - 07:08 JuanCarlosGuerra  
camera_lens_move20220315.pngpng camera_lens_move20220315.png manage 97 K 16 Mar 2022 - 00:49 JuanCarlosGuerra  
ccd47_3p30.pngpng ccd47_3p30.png manage 47 K 16 Mar 2022 - 10:41 JuanCarlosGuerra  
ccd47_acq_12p51.pngpng ccd47_acq_12p51.png manage 27 K 16 Mar 2022 - 07:52 JuanCarlosGuerra  
gcs_3p39.pngpng gcs_3p39.png manage 625 K 16 Mar 2022 - 10:40 JuanCarlosGuerra  
jitter_esm_03p20.pngpng jitter_esm_03p20.png manage 124 K 16 Mar 2022 - 10:20 JuanCarlosGuerra  
ncap_on_off_20220316.pngpng ncap_on_off_20220316.png manage 737 K 16 Mar 2022 - 21:24 JuanCarlosGuerra  
optimize_12p26.pngpng optimize_12p26.png manage 206 K 16 Mar 2022 - 07:26 JuanCarlosGuerra  
optimize_2p20.pngpng optimize_2p20.png manage 219 K 16 Mar 2022 - 09:20 JuanCarlosGuerra  
optimize_esm_3p20.pngpng optimize_esm_3p20.png manage 179 K 16 Mar 2022 - 10:22 JuanCarlosGuerra  
Topic revision: r3 - 19 Oct 2022, JuanCarlosGuerra
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback