Included in Software/LBCControlSoftwareNotes page

LBC Software Updates

Binocular/Queue
  • Checking authorized or just enabled via the LBC GUI?
  • LBC is the only instrument that uses the IIF BOTH - probably still need it, since we don't want to change LBC to send two presets.
    TCS always treats BOTH as async.
    • if both sides are selected in LBC GUI (or is that in the OB?)
    • SYNCOFFSET added to OB to support Q scripts - TCS binoc sync offset sent if NOT both AND sync is set in OB (defaults in old OBs to true).
  • I have a note that says "If you have both LBCs authorized and connected to TCS, but only send a preset from one side, it will not go through on the TCS side, because it waits to be sync'ed with the other side."
    That must apply to mixed mode? Otherwise, how would you get a preset from only one side? LBC would send one preset as BOTH .
  • LBC has this bogus convention of not sending preset to TCS if the DEC=-90 - kludge in ActionPlay function of actions.c, doesn't call TCSPresetStart if DEC > -89.
  • LBC needs to fail when TCS commands fail - if not, it won't bubble up to the Q software
  • not checking the return status of the function TCSMotionWait . It is called in a PTHREAD_CREATE

General
  • FACSUM guiding status from LBC can be incorrect - LBC software is not taking into account the fTelescope flag when setting the string it sends to FACSUM, so it can report that it's guiding when it's not really sending anything to TCS.
  • Non sidereal tracking? According to Stefano Gallozzi, the initial developer of guiding:
    "Moreover if you plan to support the non sidereal tracking it is difficult you can track an object, with different movements respect to stars, using only tech-chips (as the tracking does).
    It is critical you identify a new routine in the science control system which updates the camera/telescope movements in real time directly on the scientific-chips superimposing the shifts to the open loop sidereal tracking (not the guiding).
    This means that this future routine will switch-off the standard guiding software. "
    But, John says PCS changes have been made since then to make it work in LBC. See IT 7387
  • INDI properties are not being published
  • when HK fails (both sensors timeout - IT 4236), it never recovers and actually hangs (maybe should take out the retry logic?) - I think it actually can cause things to lock up - while testing out of shutdown 2014, we saw this instance 20140905, 18:29 - loads of watchdog on StatusThread started a new instance and no response on gui
    Another instance of the watchdog complaints occurred in May-2015 (see ITs 5559 and 5561) when TCS locked up many times. I think the semaphore is stuck because it spawns a new thread, but it cannot do anything, just waits on the semaphore. I made a test program that just did a sleep in the thread - that's enough to make the sem_trywait return and make it think it needs to cancel the thread and restart.
    Process monitoring, or heartbeats would help - cannot tell when housekeeping hangs
  • When we get sensor timeouts on HK we should be able to recover in software since it is fixed with just an lbckill/lbcstart but I've tested just closing/opening the socket and it doesn't work.
  • It would be nice to have a "reset" that power-cycles the electronics, but not the windows PCs.
  • the load of the rotator trajectory sometimes takes about 5 secs longer than it should - we see no delay, late by xxx secs messages
    is it the load taking a long time, or something else?
  • Cannot use a firefox browser on the CMU to send commands - it passes an extra colon in the name, so the parsing in lbc.c fails and says "unknown command" :
    2016/10/06 20:17:54.895906 N -      unknown command >:1:TURNONHK 1 1<  
  • one-sided turn on complains about the other side (tried to fix it in channel.c but it wouldn't turn on at all - complaining "off by command" or something.
  • CCD code error handling not initializing?
  • IDL init robustness
  • TRACKERS not recognizing a STOP from channel after too few/too many recovery?
  • Make sensor timeout a different value so its alarm can be different?
  • if TCS ON fails on recover (from power.conf), it won't try any of the other startup. Is that ok? maybe TCS ON should be last? See 20170907 20:46.
  • Errors in the httpd log
  • Find a more clever way to set Z4 value for the GUI - it's zero'ed on preset and expose so you rarely see the value non-zero; it needs to happen on the first exposure of a filter set, not each exposure. Can I make an interface so that the channel-level, where it knows the OB steps, can clear the Z4 correction value?
    Should we be displaying the "delta" value instead?
    The tech code knows when it's the first one, doesn't it? Can we use that flag?
  • is there a race condition sometimes between bringing up HK and checking inhibited (see channel code line 896)
  • could we clear the inhibit counter on TURN OFF? see IT 5319
  • when command fails/rejected, it would be useful to see the command name; for instance on an UNKNOWN failure like when the system is in the wrong state for the command
  • add integrator to rotator tracking
  • update the CAN doc for swapping in a new Windows PC because there is no PCI card in the spare
  • Configure the upsmon to turn off systems and HK before poweroff.
    Created lbcoff.sh script that is used by the LBC shutdown script in /etc/rc0.d
GUI
  • Add an option to NOT archive files? Force a GUI pop-up if we do, similar to what it does when you don't enable telescope.
  • error on power line can lead to a deactivated subystem and NO options available on the GUI
  • we had an instance where the red CAMERA didn't initialize which caused a TURNON failure and the system went through the uninit, but red never completed the uninit, so the GUI wouldn't let us do anything
  • Olga said she's seen the PLAY button available when the rotator is still positioning
  • fast OB generation should default to Telescope and Guiding and Keep Current Pointing according to Dave T
  • add "move to target" option on fast OB generator so they don't have to STOP an OB, just issues preset
  • The fast OB generator automatically adds "clear" to the list of filters for both red and blue in the drop down list that is built from the filters file in the conf directory. If you select "Target type" of "dark" or "zero", it puts "clear" in for the filter name. This is translated in the instrument control software to the empty position.
    According to JMH: The "clear" filter (it does exist) hasn't been in LBC-Blue since 2006. I don't think that LBC-Red has ever had the "clear" filter installed.
  • CCD temp on GUI should be --- when the camera shuts down so it's not deceiving
  • How is pause an OB different than stop? Is this tied up in the OBExecutionConfigFile ? this file must be written when you pause an OB (see actions.c )
    OBExecution = TRUE 
    OBFileName = lbc.070516.182516.ob 
    OBFullFileName = /home/lbccontrol/obs/lbc.070516.182516.ob 
    CompletedImages = 4 
    CompletedDitheringSteps = 0 
    CompletedBlueFilters = 0 
    ActualFilterCompletedBlueImages = 4 
    CompletedRedFilters = 0 
    ActualFilterCompletedRedImages = 0
Image Analysis
  • is there a memory leak in error handling of the IA tracking code? like when it doesn't find a file?
  • change track_first to let you know the guide star it chooses
  • archive the OB files like we do logs - this will also require us to save the old logs/OBs somewhere other than the CMU because it is not backed up
    this is on the /lbt/data/logs partition now, mounted on the CMU.
RPC/CCD Controllers
  • What is RPC error 104?? - it looks like it's a "reset by peer", a hangup
    while testing the upgrade, I had the Windows PC up - when the RPC server process crashes on the Windows side, we get a 104 and cannot recover in the retry count
  • scrub the code for the error-handling so that the save is not interrupted? we've had TRACKER RPC errors interrupt a science image save, etc.
  • the test IBM PC is running Windows XP, we should load Windows 2003; make sure to load drivers for network
  • To track down RPC errors: release go.bat script that logs when restarting the RPC servers; redirect Windows side output; check all timeouts are long enough; send DD entries in separate thread?
  • we should never see the RPC error due to PC startup delay message in the log because to even decide the power is on, it does a clnt_create - why do we ever see this message?
  • RPC error recovery cannot tell when the CCD RPC process actually crashes - if it does, and is restarted (automatically), the CCD controller is not initialized. The ErrorCheckAndRecovery function is only for CCD controller errors - not RPC error recovery.
  • does the CCD controller process on the windows PC sometimes crash during disk cleanup? which then causes RPC errors afterward
Test Programs
  • testfilters should let you startup the system again if fails during init; why does testfilters set the "new" flag for configuration so that it writes and complains on shutdown?
  • allow testrotator to have absolute angle commands, without auto-unwrapping:
    Subject:    absolute angles in test rotator?
    Date:    Thu, 11 Sep 2014 15:17:15 -0700
    From:    John Hill <jhill@as.arizona.edu>
    To:    ksummers@lbto.org
    
    Kellee,
       Have you thought about whether it would be possible to
    give absolute angle commands in testrotator?  By absolute, I mean
    angles that are not automatically unwrapped.  Presently it applies
    unwrap algorithms to whatever command you send, so that you can only
    explore -270 to +89 and not the whole range of the rotator. The range
    is slightly different for red and blue presumably because they have
    slightly different zeropoints.
    
    I don't have any use for this other than making testrotator a bit
    more useful for hardware and cable tests.
                      Ciao,
                         John

  • testfilters fails trying to uninitialize when the system has an error and uninitializes on its own - need to check before blinding calling uninit to shutdown
  • make testrotator capture ^C and stop tracking
  • Updates for Windows CCD controller test program
    • has to start in lbc\bin to find the sci headers
    • should include B/R on FITS names
    • should wait for CR for all inputs, not just some
  • GUI problems with checking/unchecking systems for you? once I had all blue unchecked and when I clicked "Turn ON", they all checked! also once I saw the GUI change my "connect LBT" check boxes.
    The GUI will update to the last saved values every 5 minutes - if you've changed the checkmarks, but not submitted (like push "Turn On", or "Play", etc), then the values will go back to what they were for the last submission.
  • I'd like to change the log rollover time - hardcoded in lbc.c; it's using noon (machine time, which is UTC) so that's 5am. If it used 00:00UTC instead, it would be 5pm local time.
    Andrea and JohnH didn't like 00:00UTC. Changed to noon local time after Dave T requested that time.
Topic revision: r25 - 13 Sep 2018, KelleeSummers
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback