W start up. Ok.

Secondary start up :ok.
second startup, problem with the secondary flatAO, the set up process stop. I

from the log analysis:

HOUSEKPR00.00001254807705.log

HOUSEKEEPER |ALW| 536330|2009-10-06 07:33:37.725545| CONFIG-LOADER > My level is 6 (INF)
HOUSEKEEPER |INF| 536331|2009-10-06 07:33:37.725560| CONFIG-LOADER > Getting configuration file '/usr/local/adopt/conf/left/mirrorctrl_adsec.conf'
HOUSEKEEPER |INF| 536332|2009-10-06 07:33:37.725607| CONFIG-LOADER > LBT SIDE: L
HOUSEKEEPER |INF| 536333|2009-10-06 07:33:37.725895| CONFIG-LOADER > Dict of size 31 created from conf/left/mirrorctrl_adsec.conf
HOUSEKEEPER |INF| 536334|2009-10-06 07:33:37.725952| MAIN > Reading DSP [255-255] from BCU_SWITCH [HouseKeeper.cpp:2161]
HOUSEKEEPER |DEB| 536335|2009-10-06 07:33:37.725966| MAIN > CommandSender::sendMultiBcuCommand: SEQ request
HOUSEKEEPER |DEB| 536336|2009-10-06 07:33:37.725976| MAIN > CommandSender::sendMultiBcuCommand: allocating shm...
HOUSEKEEPER |DEB| 536337|2009-10-06 07:33:37.726013| MAIN > CommandSender::sendMultiBcuCommand: shm allocated !!!
HOUSEKEEPER |DEB| 536338|2009-10-06 07:33:37.726032| MAIN > sendMultiBcuCommand: shared memory cleaned
HOUSEKEEPER |DEB| 536339|2009-10-06 07:33:37.726078| MAIN > sendMultiBcuCommand: REPLY EXPECTED !!!
HOUSEKEEPER |ERR| 536340|2009-10-06 07:33:38.728483| MAIN > Error reading BCUs: [CommandSenderTimeoutException] 06/10/2009 07:33:38.728214 Timeout expired [HouseKeeper.cpp:2186]
...

HOUSEKEEPER |INF| 536499|2009-10-06 07:34:07.790016| MAIN > Reading DSP [255-255] from BCU_SWITCH [HouseKeeper.cpp:2161]
HOUSEKEEPER |DEB| 536500|2009-10-06 07:34:07.790031| MAIN > CommandSender::sendMultiBcuCommand: SEQ request
HOUSEKEEPER |DEB| 536501|2009-10-06 07:34:07.790041| MAIN > CommandSender::sendMultiBcuCommand: allocating shm...
HOUSEKEEPER |DEB| 536502|2009-10-06 07:34:07.790067| MAIN > CommandSender::sendMultiBcuCommand: shm allocated !!!
HOUSEKEEPER |DEB| 536503|2009-10-06 07:34:07.790087| MAIN > sendMultiBcuCommand: shared memory cleaned
HOUSEKEEPER |DEB| 536504|2009-10-06 07:34:07.790118| MAIN > sendMultiBcuCommand: REPLY EXPECTED !!!
HOUSEKEEPER |ERR| 536505|2009-10-06 07:34:08.792609| MAIN > Error reading BCUs: [CommandSenderTimeoutException] 06/10/2009 07:34:08.792530 Timeout expired [HouseKeeper.cpp:2186]
HOUSEKEEPER |ALW| 536506|2009-10-06 07:34:09.793596| CONFIG-LOADER > My level is 6 (INF)
HOUSEKEEPER |INF| 536507|2009-10-06 07:34:09.793622| CONFIG-LOADER > Getting configuration file '/usr/local/adopt/conf/left/mirrorctrl_adsec.conf'
HOUSEKEEPER |INF| 536508|2009-10-06 07:34:09.793674| CONFIG-LOADER > LBT SIDE: L
HOUSEKEEPER |INF| 536509|2009-10-06 07:34:09.793971| CONFIG-LOADER > Dict of size 31 created from conf/left/mirrorctrl_adsec.conf
HOUSEKEEPER |INF| 536510|2009-10-06 07:34:09.794032| MAIN > Reading DSP [255-255] from BCU_SWITCH [HouseKeeper.cpp:2161]
HOUSEKEEPER |DEB| 536511|2009-10-06 07:34:09.794046| MAIN > CommandSender::sendMultiBcuCommand: SEQ request
HOUSEKEEPER |DEB| 536512|2009-10-06 07:34:09.794056| MAIN > CommandSender::sendMultiBcuCommand: allocating shm...
HOUSEKEEPER |DEB| 536513|2009-10-06 07:34:09.794080| MAIN > CommandSender::sendMultiBcuCommand: shm allocated !!!
HOUSEKEEPER |DEB| 536514|2009-10-06 07:34:09.794099| MAIN > sendMultiBcuCommand: shared memory cleaned
HOUSEKEEPER |DEB| 536515|2009-10-06 07:34:09.794129| MAIN > sendMultiBcuCommand: REPLY EXPECTED !!!
HOUSEKEEPER |INF| 536516|2009-10-06 07:34:09.794838| MAIN > Reading DSP [255-255] from BCU_MIRROR_ALL [HouseKeeper.cpp:2169]
HOUSEKEEPER |DEB| 536517|2009-10-06 07:34:09.794852| MAIN > CommandSender::sendMultiBcuCommand: SEQ request
HOUSEKEEPER |DEB| 536518|2009-10-06 07:34:09.794900| MAIN > CommandSender::sendMultiBcuCommand: allocating shm...
HOUSEKEEPER |DEB| 536519|2009-10-06 07:34:09.794917| MAIN > CommandSender::sendMultiBcuCommand: shm allocated !!!
HOUSEKEEPER |DEB| 536520|2009-10-06 07:34:09.794938| MAIN > sendMultiBcuCommand: shared memory cleaned
HOUSEKEEPER |DEB| 536521|2009-10-06 07:34:09.794960| MAIN > sendMultiBcuCommand: REPLY EXPECTED !!!
HOUSEKEEPER |INF| 536522|2009-10-06 07:34:09.795755| MAIN > Reading DSP [0-27] from all BCUs [HouseKeeper.cpp:1903]
etc. etc. restarting reading properly housekeeping data after about 30sec, but with a FUNCTALARM:

HOUSEKEEPER |ERR| 536623|2009-10-06 07:34:09.832259| FUNCTALARM > FunctAlarm DSPPosCommand-0276 nan

========================== ========================== FASTDGN00.00001254812347.log

FASTDGN00 |INF| 17503|2009-10-06 07:33:35.888934| FAST > Processed 50 cycles of 7637 vars in 0.996 s ( 50.2 Hz). HW frame rate 50.2 [DiagnApp.cpp:720]
FASTDGN00 |INF| 17504|2009-10-06 07:33:36.909137| FAST > Processed 51 cycles of 7637 vars in 1.02 s ( 50 Hz). HW frame rate 50 [DiagnApp.cpp:720]
FASTDGN00 |INF| 17505|2009-10-06 07:33:38.951207| ADAM > Adam: disabling coils...
FASTDGN00 |DEB| 17506|2009-10-06 07:33:38.951247| ADAM > Command to Adam: #011300
FASTDGN00 |DEB| 17507|2009-10-06 07:33:39.051245| ADAM > [UdpTimeoutException] 06/10/2009 07:33:39.051202 UdpConnection: timeout receiving data (TIMEOUT_ERROR)
FASTDGN00 |ERR| 17508|2009-10-06 07:33:39.051260| ADAM > Adam: coils disabling failed
...

FASTDGN00 |DEB| 17534|2009-10-06 07:33:46.658349| ADAM > Command to Adam: #011300
FASTDGN00 |DEB| 17535|2009-10-06 07:33:46.758340| ADAM > [UdpTimeoutException] 06/10/2009 07:33:46.758292 UdpConnection: timeout receiving data (TIMEOUT_ERROR)
FASTDGN00 |ERR| 17536|2009-10-06 07:33:46.758356| ADAM > Adam: coils disabling failed
FASTDGN00 |INF| 17537|2009-10-06 07:33:47.759301| ADAM > Adam: disabling coils...
FASTDGN00 |DEB| 17538|2009-10-06 07:33:47.759328| ADAM > Command to Adam: #011300
FASTDGN00 |DEB| 17539|2009-10-06 07:33:47.759909| ADAM > Answer from Adam: 01^M#011300
FASTDGN00 |INF| 17540|2009-10-06 07:33:47.759920| ADAM > Adam: coils succesfully disabled

============================ ============================ No FSM/IDL commands have been sent while the mirror was lost for 30seconds (see the following lines of IDLCRTL log)

IDLCTRL00.00001254812123.log

IDLCTRL00 |DEB| 720|2009-10-06 07:01:23.823629| MAIN > Message 36864
IDLCTRL00 |TRA| 721|2009-10-06 07:01:23.823644| MAIN > 14 bytes received from ADSECARB00 - seq:132972
IDLCTRL00 |TRA| 722|2009-10-06 07:01:23.823654| MAIN > Command was: print, errcode
IDLCTRL00 |ALW| 723|2009-10-06 07:01:23.823756| MAIN > StartCmd

IDLCTRL00 |ALW| 724|2009-10-06 07:01:23.823783| MAIN > 0

IDLCTRL00 |ALW| 725|2009-10-06 07:01:23.823800| MAIN > EndCmd

IDLCTRL00 |DEB| 726|2009-10-06 08:48:11.625603| MAIN > Message 36864
IDLCTRL00 |TRA| 727|2009-10-06 08:48:11.625634| MAIN > 35 bytes received from ADSECARB00 - seq:139355
IDLCTRL00 |TRA| 728|2009-10-06 08:48:11.625693| MAIN > Command was: errcode = fsm_fault_recovery(/AUTO)
IDLCTRL00 |ALW| 729|2009-10-06 08:48:11.625888| MAIN > StartCmd
IDLCTRL00 |ALW| 730|2009-10-06 08:48:11.625907| MAIN >

IDLCTRL00 |ALW| 731|2009-10-06 08:48:12.347528| MAIN > FAULT identification: try to read from all CrateBCUs

IDLCTRL00 |ALW| 732|2009-10-06 08:48:12.347587| MAIN > % Compiled module: FSM_FAULT_RECOVERY.

The sequence of the faults is the following: ================================= ---++++++ SECONDARY BACK AGAIN......... ================================= ---++++++ TEST CLOSE LOOP BINNING 1 HIGH FLUX GAIN :0.0 AT DIFFERENT FREQUENCIES: 1000,800,625 APPLYING DISTURBANCES.

CLOOP <20091006_154157
_154306>

>
Bin = <1>
LoopHz = <800>
Pup = <>
Dist =<dist_atm_s0.8_L040.0_v15.0_OVfreq1000.00_np4000.fits>
Gain = <gain =0>
Rec = <20090925_151900>
SN = <zero
FW1 = <700-1000>
FW2 = <800-10>
Phot = <aprox 2500> phot/subap
Nota = <> fsm_set_flat does not longer working properly. At the beginning the setting process the coils are disable. CLOOP <20091006_154614
_154719>

>
Bin = <1>
LoopHz = <625>
Pup = <>
Dist =<dist_atm_s0.8_L040.0_v15.0_OVfreq625.00_np4000.fits>
Gain = <gain =0>
Rec = <20090925_151900>
SN = <zero
FW1 = <700-1000>
FW2 = <800-10>
Phot = <aprox3000> phot/subap
Nota = <> The problem seems to be associated to communication drop: CLOOP <20091006_154956
_155206>

>
Bin = <1>
LoopHz = <1000>
Pup = <>
Dist =<dist_atm_s0.8_L040.0_v15.0_OVfreq1000.00_np4000.fits>
Gain = <gain =0>
Rec = <20090925_151900>
SN = <zero
FW1 = <700-1000>
FW2 = <800-10>
Phot = <aprox 1900> phot/subap
Nota = <> in order to reduce the problem THE ADAM AND BCUs WATCHDOGS have been DISABLED. DARK: CLOOP <20091006_155819
_155934>
=========================== ---+++ BIN 4 Rec_20091002_150920 , 6L/D, modes 35 The mirror restarted working. Test of the close loop of bin 4. Please, report in case the mirror stops again after the last restart. good up to gain0.5
Saved close loop diagnostics + IRTC

CLOOP <20091006_160916 >
_161146>
Bin = <4>
LoopHz = <200>
Pup = <>
Dist =<off>
Gain = <gain =0.4>
Rec = <20091002_150920>
SN = <zero
FW1 = <700-1000>
FW2 = <800-10>
Phot = <aprox 2200> phot/subap
Nota = <IRTC EXPOSURE TIME AS BEFORE CLOSE LOOP CAN BE USED THE DARK : CLOOP <20091006_155819, _155934> -- JuanCarlosGuerra - 2009-10-06 CLOOP <20091006_162003>
_162119>
Bin = <4>
LoopHz = <200>
Pup = <>
Dist =<_ovfreq625>
Gain = <gain =0.7>
Rec = <20091002_150920>
SN = <zero
FW1 = <50-50>
FW2 = <800-10>
Phot = <aprox 60> phot/subap
Nota = <LAMP 13, IRTC EXPOSURE TIME AS BEFORE. THE DARK : CLOOP <20091006_162316, also it has been saved a irtc dark acquire.check in the directory /calib/..

-- JuanCarlosGuerra - 2009-10-06
Topic revision: r6 - 07 Oct 2009, JuanCarlosGuerra
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback