YYYY-MM-DD - test gain ramp failure

Software: BM
AO:

Software versions used: stable, test.YYYYMMDD.reorder-gain-ramp
Software version after test: stable

Test 1: Gain ramp IDL procedure

Servers and software versions used:
sxadsec: test.YYYYMMDD.reorder-gain-ramp
soul-sxwfs: stable (no change)

Problem: The previous addition of the gain ramp performed ramping in the order of: 1) set gain (ramped), 2) set gain (original), 3) set AdSec state to AORunning. If it completed successfully, the AdSec Arbitrator state would also advance to AORunning. Jenny noted in the nightly report for 2022-02-20 that this should be reordered so that the state changes before the gains are updated. However, this could result in a problem if the second set gain fails (such as from a communication timeout) before the state is updated, because the AdSec and AdSec Arbitrator states could remain in LoopSuspended with non-zero gains if the second set gain were to fail.

New code: (GH #106)
Xianyu's updated IDL procedure takes care of this by first attempting to set the gains back to zero if the fsm_set_gain fails, and if this also fails, causing the shell to RIP. If setting the gains back to zero succeeds, it will return an IDL_GAIN_RAMP_ERROR, which the AdSec Arbitrator will then respond to by moving to a failure state, which a person can then manually recover from as necessary (Panic, or in the future, recover fail.)

Test: To test that the gain ramp works as intended, we just need to close the loop and execute Pause and Resume commands as normal, while tailing the IDL logs to see that the command execution is working. We may also want to record optical loop data to verify that the frames are being affected by the scaled gains.
  1. To test if the error handling performs properly for the condiiton that the first fsm_set_gain fails, we can create a FITS file with the incorrect number of modes, and before a PauseAo, update the RTDB variable ADSEC.L.G_GAINS_A to the new FITS filename. The number of modes are checked at the beginning of fsm_set_gain, so this will cause the first fsm_set_gain to fail, which should cause fsm_resume_ao to set the gains back to zero and the AdSecArbitrator to update the state to Failure.
    1. This does not test the case that the second fsm_set_gain fails. Open to suggestions if we feel it's necessary to test this as well.
Topic revision: r8 - 03 May 2022, BrandonMechtley
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback