LBTO Telemetry

LBTO uses HDF5 files to record telemetry streams captured from the telescope into a permanent location. The library is used by various services (TCS, DIMM, OVMS, OAC, LUCI). Current library version is stored at /lbt/telemetry.

The following terms are used:

  • stream is a set of channels. When a new record is added to the stream, all channels values are recorded
  • channel is a value to be recorded

Where are data stored

Data are stored into /lbt/data/telemetry directory. That convention is enforced by programs linking to the telemetry library (e.g. tcs stores path as TEL_STORE_DIR in tcs/etc/tcs.etc configuration file). HDF5 files are rsynced every morning (13:45 UT / 6:45 MST for TCS, 14:30 UT / 7:30 MST for OVMS) to the downtown fileserver, and available then directly from downtown machines. See HDF5TelemetryDataMaintenance for how are the data replicated.

All telemetry files are expected to contain a single dataset, names _01. HDF5 telemetry allows storing of records as channels entries, this is used at least for PMC actuator telemetry.

Caveats

  • The telemetry library writes to the system SYSLOG, so always check there for errors.
  • Telemetry uses a leap-seconds.list file. This file is provided by NIST. In early June and December get the next version of the file from ftp://ftp.nist.gov/pub/time/leap-seconds.list and copy it to every place we're running telemetry. ( See README.md)
  • If telemetry format (defined in C++ source code) is changed, and the *current link for the affected stream exists, the library will start *_02 dataset in existing file (will switch back to *_01 once new file is created). Remove *current file to prevent this behaviour.

Notes / Issues

  • Complaints like The sample with time NNNN MJD-TAI(s) was received out of order. are seen when telemetry gets samples with the same timestamp - check that the timestamp is using precision better than seconds.
  • Messages like Proxy hasn't been bound to a buffer typically mean that a 'store' is called on a proxy that has not been put into a stream with an 'add_child'.

What's logged

The following list is for reference only. For full list, please see the telemetry visualization map.

  • LUCI
    • dewar temperature and pressures

  • OAC
    • housekeeping - CCD temperatures, ambient temperatures, ..

  • TCS
    • ECS
      • chambre tempertures
      • shutters and vents end sensors
      • mirrors ventilation
    • GCS
      • calculated guiding offsets
      • collimation (tip/tilts demands)
      • WFS Zernikes
    • MCS
      • HBS pressures, temperatures, ...
    • MCSPU
      • servos demands
      • encoders readouts
    • PCS
      • mountequatorial - target RA, DEC, polar angles (@20HZ, the fast loop)
      • sides tip/tilts
      • offset and guide arguments
      • DUT1, earth axis orientation
    • PMC
      • acuators state
      • actuators commanded position, expected position,..
      • mirror temperature
    • PSF
      • tip/tilts demands (PCS + GCS / WFS)

Source code

C++ source code is stored on lib-telemetry GitHub. There are plans to transition telemetry code to C++11 and make the library available as rpm packages. Please see README.md and CHANGELOG.md for details (how to compile, how to use the library).

Subsystems notes

TCS Subsystems

The global TCS flag collectTelemetry can be set in the file: /lbt/tcs/current/tcs/etc/tcs.conf to enable/disable telemetry for all TCS subsystems .
Likewise, the path to the HDF5 files is in the same file as TEL_STORE_DIR . (doesn't look like the subsystems have their own flag for the directory path)

Also, each subsystem has its own flag

%CODE{lang="sh"}% ./2014C/iif/etc/iif.conf IIFtelemetry bool true ./2014C/ecs/etc/ecs.conf ECStelemetry bool true ./2014C/gcs/etc/gcs.conf GCSL.use_telemetry bool true ./2014C/gcs/etc/gcs.conf GCSR.use_telemetry bool true ./2014C/oss/etc/oss.conf OSStelemetry bool true ./2014C/mcs/etc/mcs.conf mcsTelemetryFlag bool true ./2014C/mcs/etc/mcs.conf mcspuTelemetryFlag bool true ./2014C/pmc/etc/pmc.conf PMCtelemetry map<int,bool> 0:true 1:true ./2014C/psf/etc/psf.conf PSFtelemetry map<int,bool> 0:true 1:true ./2014C/pcs/etc/pcs.conf PCSTelemetry bool true ./2014C/env/etc/env.conf ENVtelemetry bool true %ENDCODE%

MCSPU

MCSPU uses a separate /lbt/tcs/current/tcs/etc/tcs.conf file

OVMS

OVMS telemetry is controlled via two processes - a server, run as a daemon on the ovms host and a client that stops/starts ( pause or resume ) collection. Daily cron jobs pause telemetry in the morning and restart it in the evening.

The OVMS system is a 64-bit system but the compiler is a different version than the TCS system, so the collection library is built on the ovms host. See Software/OVMSSoftwareBuildAndInstall#Telemetry for step-by-step directions.

DIMM

DIMM uses the telemetry collection library directly as of May-2015. The computer is CentOS 4.8 machine, so it needs its own copy of the source.
See Software/DIMMSoftwareBuildAndInstall for building DIMM with the latest telemetry source.

OAC

OAC uses the telemetry collection library. The mountain OAC computer is CentOS 6.5, 32-bit machine, it needs a 32-bit copy of the library (same version as mcspu).

Topic revision: r7 - 18 Oct 2018, PetrKubanek
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback