2022-04-19 - bcu2k setup timeouts: test agw_power fix that waits for RelayCtrl to be in OPERATING state before issuing power on commands

  • Time: daytime
  • Software: BM
  • AO:
  • Software versions used:
    • soul-sxwfs: test.20220419.agw-power-wait-for-operating-rebase-onto-stable
  • Software version after test:
    • soul-sxwfs: test.20220419.agw-power-wait-for-operating-rebase-onto-stable

Test: leave fix for agw_power in place on soul-sxwfs

Background: There are frequently timeouts during turnon for different agw_power devices (e.g. PROG_BCU2K, PROG_BCU47, RESET_BCU2K, RESET_BCU47). I traced this in the logs to the commands being refused by the RelayCtrl process (AbstractRelayCtrl) because the state is not STATE_OPERATING. This means that when it happens, the RelayCtrl has just not had enough time to initialize its serial connection yet.

New code: (Files changed)
agw_power.py: before sending the command to turn on a device, the WFS arbitrator (agw_power) will first make sure the RelayCtrl state is STATE_OPERATING, and if it's not, will wait up to 10s for it to become so.
AbstractRelayCtrl.cpp: this also modifies the RelayCtrl and PicCtrlW2 processes, which both inherit from AbstractRelayCtrl, which has bene modified to log the command name when a command is refused.

Test:
  1. BM has tested the additional logging of the refused command in AbstractRelayCtrl in a simulation VM and it behaves as expected when a command is issued outside of the STATE_OPERATING state.
  2. BM and JCG did an initial test on soul-sxwfs by executing w_start_all and executing the Operate command. Everything works as expected, but we were not able to reproduce sending a command before the RelayCtrl connection is established. JCG has recommended leaving this in place to see how it responds to future refused commands. Short of modifying AbstractRelayCtrl to allow us to force its state, this might be the best way to test it, as it will probably work OK.
-- BrandonMechtley - 19 Apr 2022
Topic revision: r1 - 25 Apr 2022, BrandonMechtley
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback