LBC Control Software

• LBC Software SPIE paper 2014, LBC Software SPIE paper 2004
• lbccontrol - all the documentation is here, hardware, software description
• http://lbccontrol.mountain.lbto.org/ui/main.html - the user interface, start here to learn how to use LBC
• http://lbccontrol.mountain.lbto.org/configuration/engineering.shtml - minicom, serial port config, etc.
• Instrumentation/LBCTroubleshooting - connectivity diagram, powering down, computer failure notes
• Full system checkout - how to run overall tests
• CCD controller test.exe
• Instrumentation/LBCWRSSoftware - new WRS replacement for DOFPIA
• current DOFPIA
• Network Overview page for access to the data probes and KVM (need to be have access to mountain network to view)
• LBC KVM - VNC to r12-kvm.mountain.lbto.org (192.168.52.86) - single-user access to each of the windows PCs and the CMU
• LBC homepage - INAF
• LBC Observers User Manual - includes lots of detailed troubleshooting
• LBC Description - replaces what was in PartnerObserving/LBCHomePage
• LBC Observing Checklists - observing wiki page for periodic checkouts
• LBC Build and Installation
• Software Rehost Status

Observing

Overview of the control interface UI in the LBC instrument documentation

Observers use power control UI to bring up the system in this order:

1. turn on housekeeping
2. click WarmUp if temp is below freezing (wait...up to about 20 minutes for the controller electronics to warm up)
3. turn on other systems with the bottom Turn ON button (status goes to initializing and then ready )
   this is powering up the windows PCs too
4. connect to LBT with top buttons
   

Observers look at the data using IDL and IRAF tools from /home/lbcobs/supportscripts/ObserverSupport and /home/lbto/iraf/LBTtools

FITS files are written to the Windows PCs in the D:\ directory and ftp'ed to the repository.
Tech FITS files are copied to /lbt/data/logs/lbc/tec/RTech and BTech in dated directories. Should be stored up to a year or so.

Observers use the scripts lbckill.sh and lbcstart.sh (aliased as lbckill and lbcstart) from the CMU to restart the control software. These have been added into the sudoers file so that they can be run as telescope and lbccontrol users.

Shutdown and Power Control

Follow the observers' shutdown order ( http://abell.as.arizona.edu/~lbtsci/Instruments/LBC/LBCUserManual.html#shutdown)

1. open the user interface http://lbccontrol.mountain.lbto.org/ui/main.html --> 'power control'
2. turn off the "other systems" (status goes to enabled )
   this powers down the windows PCs
3. if you will be powering off, then turn off "housekeeping" too
   (but in general housekeeping stays on)
4. disconnect

Note: If the GUI is active, it is the right place to power-down everything - it powers down all the components.

To just power-down the Windows PCs - have someone push the button - or telnet into the dataprobe (192.168.59.102) and turn OFF the outlets. There is no web interface to the power control with the new iBootBar. (username admin)

There is no dataprobe device for the CMU. To shut it down, run the shutdown command as telescope, lbccontrol or root

There is also a replacement to the testpower3 command line and dataprobe power control. If you run power.py as root on the CMU, you can see the power state of all the electronics. You can toggle the power from this GUI as well. If only one side is powered up, use the --channel option : power.py --channel b
Note that it does not automatically refresh the states. Use the big button at the bottom to query the power states and refresh the GUI.
Note that it is not desirable to refresh automatically because there are other programs (LBC, testpower3, dataprobe web access) querying and the dataprobe can get stuck. So it's best to manually refresh when you know nothing else is accessing the ports.

In the LBC log, every power access does a powerdump for all five of the power devices.
It represents how many users there are for each port. For example:

          powerdump: 192.168.59.113  outlets owners 20110000 last access 123 sec ago 

represents that port 1 of 59.113 (CCD controller power line) has two users (makes sense, the TRACKER and the CAMERA share the CCD controller power), port 3 has one user (HK sensors are only used by the HKEEP subsystem), and port 4 has one user (port 4 is the shutter, only the CAMERA subsystem uses that).

The housekeeping temperature and vacuum sensors share a power supply:

Machines

Requires 5 CPUs -- one Linux CPU is the Central Management Unit. The Central Management Unit (CMU) is the Linux box. It has an RPC interface to the four Windows PCs which have the CCD controllers - two for each side/color: red technical, red science, blue technical, blue science.

http://lbccontrol.mountain.lbto.org/software/linuxsoft.shtml describes the CMU setup in the LBC instrument documentation

See CMUCronJobs for details on required crons.

The LBC application has configuration files on the CMU in /lbccontrol/current/conf : redchannel.conf , bluechannel.conf (filter wheel positions, IP addresses, channel/rotator/tracker/camera configuration...), and lbc.conf (mostly paths)

For testing purposes, there is a Windows XP IBM machine. It is on the bottom shelf of one of the LBC carts on 3L, the spare power supply is on the other LBC handling cart. Jenny created a document that describes roughly how they are put back together in the lab. This document has pictures which should show exactly their location.

Software Releases

	Date	Description
CMU	Sep-2018	2.15: IT 5715 and IT 6800 updates - do dither on first offset if DEC=-90, mutex updates for error conditions in controllers
	Feb-2018	2.14a: TCS 2018A with moon almanac information
	Sep-2017	2.14: Support SYNCOFFSET flags in Q scripts; a few INDI properties published
	Sep-2016	2.13: Update for trackers stuck on STOP/RESUME and IT 5715 - wait for TCS offsets, even if -90 DEC is given
	Feb-2016	2.12: Check tec FITS image for zero in first/next instead of checking background; write guide fits thumbnail to repository dir, timestamp in DD, fix guiding status on FACSUM, fix crash in camera preset if fails 15 times.
	Jan-2016	2.11: Bug fixes to make sure we close FITS files in all cases (good and error); protect file output errors; delete AO references and files (chip1/2.gif files); cancel HK watchdog when turned OFF
	Dec-2015	2.1: Image analysis and tracking on the CMU, use target PA to calculate rotator angle, not "achieved"
	Sep-2015	2.03: Rebuilt with TCS 2015C for TCS release.
	Feb-2015	2.03: Small changes, just rolling some of the 2.1 logging, etc. fixes to the current version: - Make stopped/disabled systems WARNING instead of NOTEs. - Add function name to RPC error log messages. - Fix core when TCS is not available (NULL instrument names). - Ice version of TCSSendWavefront in /home/lbcobs/LBCFPIA. - Rollover log file at noon local time, 19:00UTC.
	Nov-2014	2.02 modified housekeeping to not "recover" when a sensor timeout occurs; increased the sleep time in power interface for new iBootBar compatibility
	Oct-2014	2.01 is 2.0 version compiled with TCS2014D and three updates for core dump fixes: TCS read/write vs command, camera preset failure recover, TURNON
	Sep-2014	New CMU version (2.0), OS is CentOS 6.5, new drivers, new PCIe profibus board, updated PHP.
	Mar-2014	Query and log the shutter motor status and positions when zero measured expo time error occurs.
	Feb-2014	Update for IT 4985 - catch a zero "measured exposure time" when non-zero was requested, stop the OB fix for trajectory load errors when STOP is requested (IT 4948) See Release1.7.txt
	Nov-2013	Updates for IT 3346 debugging; IT4315, IT4411; added OB status string to DD. For full details, see: Release1.6.txt
	Sep-2013	Populating many DD items; TCS 2013C; added more RPC error recovery; logging updates for rotator messages and RPC errors. SVN version 244 tagged as 1.5
	Aug-2013	Software updates for filter wheel mechanism changes and backlash offset; updates for ITs: 3289, 3759, 4411, 4642, 4728; updates for sending DD entries to TCS; built with TCS 2013B release; documentation and logging updates. For full details, see: Release1.4.txt
	26-April-2013	Added 5sec sleep in housekeeping when resetting before checking for bad reading again; Clear fStop flag when starting a new command - fix for IT 4316 documentation updates
	26-March-2013	Red HK Use it should be gray when HK is ON, like Blue HK ; web fixes for some errors, typos Added FILE, LINE logging in macros, more logging in HK on restarts, delete unused variables; Updates for balance function and to make testfilters not have to be restarted to run a different function; documentation updates
	Sep-2012	Updates to config for new CCD controller hardware ID; delete Balluff device; CMU software built by KS; checkin of software updates by Andrea DiPaola for the ccd controller software on the windows machines
	Oct-2011	Last version built and released by Andrea DiPaola.
Windows software	Aug-2018	Updates to offset files in lbc.zip for all controllers (Fernando and Gustavo)
	Aug-2017	Built on Windows 7 virtual machine, deleted image analysis/tracking executables
	Mar/Apr-2017	Updated offset file for blue science (see ITs 6542 and 6549)
	Jan-2016	Save the FITS files for "too few" or "too many" downloaded pixels
	Sep-2014	Log shutter open telemetry on failure.
	Mar-2014	Don't check for background save errors during preset or expose - no reason to abort. Also released Baruffolo's source/my build of lbciaRun.exe .
	Feb-2014	Check for zero measured exposure time and return SHUTTER error to user.
	Nov-2013	Sept-2013 lbc.zip released: fix for stale guide image (IT 4411) when track_first fails; update to all chips bias values from summer shutdown checkout; LBTO build of all DiPaola source. Baruffolo's source of lbciaRun.exe not released because lbcia failed during startup testing (we ran with it during startup until it was reverted during science)
	Sep-2013	new source from Gallozzi for track_first and track_next - built by LBTO (not part of lbc.zip so this is released)
	29-Aug-2012	Last version built and released by Andrea DiPaola

Software Description

See this page for details of the Windows/Linux interactions, what software lives where, third-party stuff used, etc.

Software Updates

Binocular/Queue

• Checking authorized or just enabled via the LBC GUI?
• LBC is the only instrument that uses the IIF BOTH - probably still need it, since we don't want to change LBC to send two presets.
  TCS always treats BOTH as async.
  • if both sides are selected in LBC GUI (or is that in the OB?)
  • SYNCOFFSET added to OB to support Q scripts - TCS binoc sync offset sent if NOT both AND sync is set in OB (defaults in old OBs to true).
• I have a note that says "If you have both LBCs authorized and connected to TCS, but only send a preset from one side, it will not go through on the TCS side, because it waits to be sync'ed with the other side."
  That must apply to mixed mode? Otherwise, how would you get a preset from only one side? LBC would send one preset as BOTH .
• LBC has this bogus convention of not sending preset to TCS if the DEC=-90 - kludge in ActionPlay function of actions.c, doesn't call TCSPresetStart if DEC > -89.
• LBC needs to fail when TCS commands fail - if not, it won't bubble up to the Q software
• not checking the return status of the function TCSMotionWait . It is called in a PTHREAD_CREATE

General

• FACSUM guiding status from LBC can be incorrect - LBC software is not taking into account the fTelescope flag when setting the string it sends to FACSUM, so it can report that it's guiding when it's not really sending anything to TCS.
• Non sidereal tracking? According to Stefano Gallozzi, the initial developer of guiding:
  "Moreover if you plan to support the non sidereal tracking it is difficult you can track an object, with different movements respect to stars, using only tech-chips (as the tracking does).
  It is critical you identify a new routine in the science control system which updates the camera/telescope movements in real time directly on the scientific-chips superimposing the shifts to the open loop sidereal tracking (not the guiding).
  This means that this future routine will switch-off the standard guiding software. "
  But, John says PCS changes have been made since then to make it work in LBC. See IT 7387
• INDI properties are not being published
• when HK fails (both sensors timeout - IT 4236), it never recovers and actually hangs (maybe should take out the retry logic?) - I think it actually can cause things to lock up - while testing out of shutdown 2014, we saw this instance 20140905, 18:29 - loads of watchdog on StatusThread started a new instance and no response on gui
  Another instance of the watchdog complaints occurred in May-2015 (see ITs 5559 and 5561) when TCS locked up many times. I think the semaphore is stuck because it spawns a new thread, but it cannot do anything, just waits on the semaphore. I made a test program that just did a sleep in the thread - that's enough to make the sem_trywait return and make it think it needs to cancel the thread and restart.
  Process monitoring, or heartbeats would help - cannot tell when housekeeping hangs
• When we get sensor timeouts on HK we should be able to recover in software since it is fixed with just an lbckill/lbcstart but I've tested just closing/opening the socket and it doesn't work.
• It would be nice to have a "reset" that power-cycles the electronics, but not the windows PCs.
• the load of the rotator trajectory sometimes takes about 5 secs longer than it should - we see no delay, late by xxx secs messages
  is it the load taking a long time, or something else?
• Cannot use a firefox browser on the CMU to send commands - it passes an extra colon in the name, so the parsing in lbc.c fails and says "unknown command" :

  2016/10/06 20:17:54.895906 N -      unknown command >:1:TURNONHK 1 1<  

• one-sided turn on complains about the other side (tried to fix it in channel.c but it wouldn't turn on at all - complaining "off by command" or something.
• CCD code error handling not initializing?
• IDL init robustness
• TRACKERS not recognizing a STOP from channel after too few/too many recovery?
• Make sensor timeout a different value so its alarm can be different?
• if TCS ON fails on recover (from power.conf), it won't try any of the other startup. Is that ok? maybe TCS ON should be last? See 20170907 20:46.
• Errors in the httpd log
• Find a more clever way to set Z4 value for the GUI - it's zero'ed on preset and expose so you rarely see the value non-zero; it needs to happen on the first exposure of a filter set, not each exposure. Can I make an interface so that the channel-level, where it knows the OB steps, can clear the Z4 correction value?
  Should we be displaying the "delta" value instead?
  The tech code knows when it's the first one, doesn't it? Can we use that flag?
• is there a race condition sometimes between bringing up HK and checking inhibited (see channel code line 896)
• could we clear the inhibit counter on TURN OFF? see IT 5319
• when command fails/rejected, it would be useful to see the command name; for instance on an UNKNOWN failure like when the system is in the wrong state for the command
• add integrator to rotator tracking
• update the CAN doc for swapping in a new Windows PC because there is no PCI card in the spare
• Configure the upsmon to turn off systems and HK before poweroff.
  Created lbcoff.sh script that is used by the LBC shutdown script in /etc/rc0.d

GUI

• Add an option to NOT archive files? Force a GUI pop-up if we do, similar to what it does when you don't enable telescope.
• error on power line can lead to a deactivated subystem and NO options available on the GUI
• we had an instance where the red CAMERA didn't initialize which caused a TURNON failure and the system went through the uninit, but red never completed the uninit, so the GUI wouldn't let us do anything
• Olga said she's seen the PLAY button available when the rotator is still positioning
• fast OB generation should default to Telescope and Guiding and Keep Current Pointing according to Dave T
• add "move to target" option on fast OB generator so they don't have to STOP an OB, just issues preset
• The fast OB generator automatically adds "clear" to the list of filters for both red and blue in the drop down list that is built from the filters file in the conf directory. If you select "Target type" of "dark" or "zero", it puts "clear" in for the filter name. This is translated in the instrument control software to the empty position.
  According to JMH: The "clear" filter (it does exist) hasn't been in LBC-Blue since 2006. I don't think that LBC-Red has ever had the "clear" filter installed.
• CCD temp on GUI should be --- when the camera shuts down so it's not deceiving
• How is pause an OB different than stop? Is this tied up in the OBExecutionConfigFile ? this file must be written when you pause an OB (see actions.c )
  

  OBExecution = TRUE 
  OBFileName = lbc.070516.182516.ob 
  OBFullFileName = /home/lbccontrol/obs/lbc.070516.182516.ob 
  CompletedImages = 4 
  CompletedDitheringSteps = 0 
  CompletedBlueFilters = 0 
  ActualFilterCompletedBlueImages = 4 
  CompletedRedFilters = 0 
  ActualFilterCompletedRedImages = 0

Image Analysis

• is there a memory leak in error handling of the IA tracking code? like when it doesn't find a file?
• change track_first to let you know the guide star it chooses
• archive the OB files like we do logs - this will also require us to save the old logs/OBs somewhere other than the CMU because it is not backed up
  this is on the /lbt/data/logs partition now, mounted on the CMU.

RPC/CCD Controllers

• What is RPC error 104?? - it looks like it's a "reset by peer", a hangup
  while testing the upgrade, I had the Windows PC up - when the RPC server process crashes on the Windows side, we get a 104 and cannot recover in the retry count
• scrub the code for the error-handling so that the save is not interrupted? we've had TRACKER RPC errors interrupt a science image save, etc.
• the test IBM PC is running Windows XP, we should load Windows 2003; make sure to load drivers for network
• To track down RPC errors: release go.bat script that logs when restarting the RPC servers; redirect Windows side output; check all timeouts are long enough; send DD entries in separate thread?
• we should never see the RPC error due to PC startup delay message in the log because to even decide the power is on, it does a clnt_create - why do we ever see this message?
• RPC error recovery cannot tell when the CCD RPC process actually crashes - if it does, and is restarted (automatically), the CCD controller is not initialized. The ErrorCheckAndRecovery function is only for CCD controller errors - not RPC error recovery.
• does the CCD controller process on the windows PC sometimes crash during disk cleanup? which then causes RPC errors afterward

Test Programs

• testfilters should let you startup the system again if fails during init; why does testfilters set the "new" flag for configuration so that it writes and complains on shutdown?
• allow testrotator to have absolute angle commands, without auto-unwrapping:

  Subject:    absolute angles in test rotator?
  Date:    Thu, 11 Sep 2014 15:17:15 -0700
  From:    John Hill <jhill@as.arizona.edu>
  To:    ksummers@lbto.org
  
  Kellee,
     Have you thought about whether it would be possible to
  give absolute angle commands in testrotator?  By absolute, I mean
  angles that are not automatically unwrapped.  Presently it applies
  unwrap algorithms to whatever command you send, so that you can only
  explore -270 to +89 and not the whole range of the rotator. The range
  is slightly different for red and blue presumably because they have
  slightly different zeropoints.
  
  I don't have any use for this other than making testrotator a bit
  more useful for hardware and cable tests.
                    Ciao,
                       John

• testfilters fails trying to uninitialize when the system has an error and uninitializes on its own - need to check before blinding calling uninit to shutdown
• make testrotator capture ^C and stop tracking
• Updates for Windows CCD controller test program
  • has to start in lbc\bin to find the sci headers
  • should include B/R on FITS names
  • should wait for CR for all inputs, not just some
• GUI problems with checking/unchecking systems for you? once I had all blue unchecked and when I clicked "Turn ON", they all checked! also once I saw the GUI change my "connect LBT" check boxes.
  The GUI will update to the last saved values every 5 minutes - if you've changed the checkmarks, but not submitted (like push "Turn On", or "Play", etc), then the values will go back to what they were for the last submission.
• I'd like to change the log rollover time - hardcoded in lbc.c; it's using noon (machine time, which is UTC) so that's 5am. If it used 00:00UTC instead, it would be 5pm local time.
  Andrea and JohnH didn't like 00:00UTC. Changed to noon local time after Dave T requested that time.

Issue Traks

IT	Detail	Status
7032	Red HK not coming up (timeout talking to dataprobe)	the iBootBar dataprobes have sometimes required someone to log in and type reboot and we haven't figured out what gets it into this state. One thing to do is to make sure we are cleaning up the socket to the device everywhere, even on errors.
6800	OB hung with R (8.0s guiding failed) and B (GOYA rotator warning)	no issue with the rotator, this problem is red controller had a hardware failure but somehow missed the STOP command that the software issued to recover. The observer issued a STOP command, but didn't wait until it was done to issue another STOP. The hang should be fixed in version 2.15, released Sep-2018.
6756	LBC pause/resume behavior	UI reported number of exposures taken as greater than total requested why is this only showing up when using one channel?
5715	stars trailed on first LBC image after a dither	John says this is likely due to LBC not checking for collimation update to finish; this is a one-sided LBC problem Should be fixed in version 2.15, released Sep-2018.
5302	Suggested changes/reorg of LBC image headers	enhancement
5205	OB progress is incorrect after pause/resume	bug
5065 5548	Timeout error on motion of filter wheel	We see this with wheel #2 for both red and blue. Seems like mostly it's a move command that doesn't get received on the motor because we timeout and then check and it hasn't moved at all.
4705	Add pointing offsets to LBC headers	enhancement Waiting on Olga for specifics, then CB can add to the TCS public interface.
4642	Add notification on failures	enhancement pop up on tracking stopped - no, we do NOT want to stop operations added ERROR message when tracking background is < 1 should it be identically zero? - cannot do a pop-up without stopping the OB. Fixed so it waits til it has tried up to the max exposure time before it complains Added Z4 to GUI. Make the background of the ? red -- needs testing. Olga also added the "shutter seems to be not working" to this IT - changed this to return an ERROR - so it does not go to Ready for the camera anymore -- tested it. how do we test the science image? don't have to, they don't have this problem
4566	Red Rotator communication problem	bug : When the rotator failed in a preset, it tried to retry indefinitely - it re-tried ~680 times before they stopped it. It is not really obvious to the operator that is happening, because it automatically retries and claims rotator system recovered from a problem ; The one error they see is: >>> preset/expose/save interrupted at preset time also core-dumped on the first lbcstart they did after this problem (this is not uncommon, often core's on startup) at least the startup core is fixed!
4236	hung at unitialization of blue HK	this is the HK timeouts - cannot seem to fix this within the software, but a lbckill/lbcstart fixes it!
3833	audible alarm on errors	enhancement
3816	Scale R/B exposures separately	enhancement
3759	Blue tracker images suddenly all zeroes	bug Happens on both sides. Goes away with an OFF/ON toggle. Sometimes it's not zero, but just funky. I have added an ERROR in the log when the background goes to zero check the fits file for zero mean (releasing Feb-2016); added a failed state to tracker status so the GUI can turn the "?" red. need to test it Also look at adding a "reset" that power-cycles the electronics, but not the windows PCs.
3595	Rotator position on first dither	still seeing problems as of Jan-2012
2765	tick marks are sticky on the UI	I did some testing - this is all controlled by the browser settings and cookies and what you submit to the GUI. If you check mark, but do not "submit" by pressing PLAY or TURN ON or TURN OFF, then the settings will reset to the last time that happened in this browser environment. If you clear and restart, nothing is sticky - if you kill and restart, it remembers. Olga notes in the IT that running dofpia and the check marks erasing? DOFPIA "submits" the http form so the checkmarks will fill in based on that.
2631	too few downloaded pixels error	This is not mine, but I am monitoring the problem.
2573	LBCB/LBCR First Image of a Sequence has small Rotator Offset (~0.5 deg.)	reopened in Sep-2013; AdP says maybe to track Parallactic angle to open shutter delay.
2546	WCS info in LBC tech chip headers	enhancement Waiting on Olga to provide
2397	RPC Client/Host Comm Error	bugs multiple problems documented here - guiding lapses, missing images, inhibit counter on restart, etc. This became a big issue coming out of summer shutdown 2013. Added better logging, retry during preset. Need to: release go.bat script that logs when restarting the RPC servers; redirect Windows side output; check all timeouts are long enough; send DD entries in separate thread?

Fernando's Requests

Task	Detail	Status
add a reset button	allow the user to reset all the electronics, but not the PCs to save time recovering can be set with config file setting WindowsPCPowerLine values for emulated
filters GUI	simple GUI over the filters test command line tool
error email	would be nice to have scripts daily grep out interesting info from the logs and send an email to Fernando/Mauro	currently sending email to myself with errors
status email	would be nice to monthly run diagnostics on the logs like we ran in January for filter performance and send an email
testfilters	add options to: move specified amount; write the current filter positions to a file
CCD test	automatically calculate bias values from the oversampling pixels and write to ini file
powertest GUI	build a simple GUI over the command line tool - for easier use and more safety	power.py created - it requires you to refresh after changing the state of something because it's not threaded - maybe sufficient?
rebuild the pc from the lab	this is the build machine for the Windows software	complete
balance check program for filters	tedious manual process is required for balancing the filters; the request is to create a simple program that runs through the commands for spinning the filter wheels and recording the current on the system for later processing	complete in lbccontrol1.2 version of testfilters

Other

Task	Detail	Status
Rehost	Replace CMU and Windows PCs with newer hardware	See the rehost notes for status
non-sidereal updates to GUI	implement changes to OB gui to access JPL horizons to look up objects and download the ephemeris for TCS
STOP failures	sometimes the STOP command actually fails and when this happens you have no recourse but to do the kill/start. But, the system is fine - it just thinks a command is running. We should make the TURNOFF available anytime to get out of this state.	AdP says I should figure out when STOP failures occur. The only thing I've found so far is comm failure (5) on a stop during a stop. Saw the same thing in Jan-2014 again (see note in IT 2397 on 3-Jan-2014).
blue tech time off?	perusing logs in May, found blue tech 5 seconds off from the other machines	15-may-2013 checked and blue tech was sync'ed Monthly, checking sync on each of the boxes
software to extract CCD controller temps from logs	to monitor whether overheating with covers on; also related to IT 4101	I have implemented tklogger to look for housekeeping errors in the lbc.log but it does not pay attention to the values - doesn't do any parsing; we should make it parse then we wouldn't get false complaints on 0 values; and we could check for CCD detector temps