INDI at LBTO

LBTO is implementing INDI (Instrument-Neutral Distributed Interface) as the mechanism for instruments to provide data to the observatory and for the status server web interface.

See the generic INDI documents:

We are using code provided by the LBTI development team. We have written several drivers for checking it out.
This page provides the details.
This page: INDIProperties, provides the details for the properties LBTO is publishing.


LBTO INDI Architecture:
StatusServerArchitectureINDI.jpg

Note that there is a subtlety to designing the INDI interfaces that will be used via the web. You don't want to just put all numbers in one property, etc. The web interface creates a callback for a device.property . So, you want to organize your properties by how they will be used in your clients.

Troubleshooting

See LBTOINDITroubleshooting

Minimal INDI

After fighting with getting enough of the full LBTI tar ball built that we could run some stuff, they created a "minimal INDI" tar ball that just has 3 libraries, the server code, a sample C++ driver, and a Python interface to getINDI.

A few makefile updates were required to set it up for LBTO's environment to use the C++ framework. It was further modified to include a few more files to use the C API.

To create a new driver, copy the indi-sample-driver directory and make one that does what you want.

The pyindi.py program can be modified to get/set/eval any INDI property. It uses the executables getINDI, setINDI, evalINDI. So make sure they're in your path if you want to use it.

The minimal INDI files have been captured in SVN in https://svn.lbto.org/view/indi/trunk/ and built/released into /lbt/tcs_devel/indi and /web/modules/INDI
To build/install required INDI libraries/include files:
 ssh tcs@tcs-test

 svn export https://svn.lbto.org/repos/indi/trunk indi
 cd indi
 gmake               (the make will complain when it tries to install the python, but that's ok, we don't need it)
 make install        (copies required files to deploy subdirectory)
 cd /lbt/tcs_devel/indi
 rm -rf *
 cp -pr ~/indi/deploy/* .  

The deploy directories should also be copied to /web/modules/INDI for use by the web server indiserver and drivers.

Simple Example

On the web machine in Tucson, we're currently running the indilbtoDDS, indiovms, indialh, and the FLAO AO drivers. To see the data published, you can do commands like:

/web/modules/INDI/bin/getINDI *.*.*
LBTO.AOStatus......
LBTO.Pointing...
LBTO.Status...
LBTO.Weather...
....

/web/modules/INDI/bin/getINDI LBTO.Weather.*
LBTO.Weather.state=1
LBTO.Weather.DewPoint=-5.0999999
LBTO.Weather.Humidity=96.900002
LBTO.Weather.Pressure=681.59998
LBTO.Weather.Rain=0
LBTO.Weather.SkyBrightness=21.059999
LBTO.Weather.Temperature=-4.6999998
LBTO.Weather.WindDir=247.59995
LBTO.Weather.WindDirF=32.599953
LBTO.Weather.WindSpeed=4.5
LBTO.Weather.WindSpeedF=7


$ getINDI ALH.*.*
ALH.Version.Driver=1.0
ALH.Version.Properties=1.0
ALH.Version.Uptime=9180.2462
ALH.AC.Guidance=OK
ALH.AC.Status=0
ALH.AO_SX.Guidance=OK
ALH.AO_SX.Status=0
ALH.AO_DX.Guidance=OK
ALH.AO_DX.Status=0
....


The following notes are about how to run the servers/drivers.

Run the server with a "driver" (this example uses the sample driver "indimaths"), logging to the local directory (dated filename.islog) - run it in the background.

ssh to tuc-shell64
export INDIHOME=/web/modules/INDI/
$INDIHOME/bin/indiserver -l ./ -vv $INDIHOME/bin/indimaths &

From another window, you can get/set properties. If you set the "x.value" property of this driver, it updates the "xmaths" properties.

$INDIHOME/bin/setINDI Maths.x.value=3

$INDIHOME/bin/getINDI Maths.*.*
Maths.Version.Driver=0.1
Maths.Version.Properties=0.1
Maths.Version.Uptime=3165.3756
Maths.x.value=3
Maths.xmaths.abs=3
Maths.xmaths.sqr=9
Maths.xmaths.sqrt=1.7320508
Maths.xmaths.value=3

When you're done, stop the server/driver:
ps -ef | grep indiserver
kill XXX     (the PID associated with your indiserver)

It may be useful to play around with, watch the log file, etc.

Here's the README file from the sample driver directory (also in indi-sample-driver )

LBTO Driver Example

Created another driver that gets the params.txt file that FACSUM and the WEATHERSTATION GUIs are reading and parses it to set some INDI parameters for us.

This driver is called inidlbtoDDS It can be run from the web or shell64 machines. The following example shows how to run a "chained" INDI server on web (with the device names and IP address of shell64 where our other server is running):
ssh to tuc-shell64
export INDIHOME=/home/ksummers/instruments/INDI/LBTI_INDI
$INDIHOME/src/INDI/indiserver -l ./ $INDIHOME/src/lbtoDDS/indilbtoDDS $INDIHOME/src/indi-sample-driver/indimaths &

ssh web@web
cd /web/statserv
export INDIHOME=/web/homes/ksummers/instruments/INDI/LBTI_INDI
$INDIHOME/src/INDI/indiserver -l ./ Maths@10.130.142.176 LBTO@10.130.142.176 &

From another window on either machine:
/web/modules/INDI/bin/getINDI *.*.*

/web/modules/INDI/bin/getINDI LBTO.weather.*
/web/modules/INDI/bin/getINDI LBTO.status.*

/web/modules/INDI/bin/setINDI Maths.x.value=100

Once the servers/drivers are running, the following web page can be run to see the current values and update the sample driver
http://statserv.lbto.org/ks/indi-demo.html


INDIPrototypeWebPage.png


I have created a TCS DDS INDI-compliant process instead of using this driver for the status server. The first test was a standalone main program that spawned an INDI thread using the port 7625. Then you can run a server that connects to it by devicename@localhost:7625. This test is in the dds directory in the /home/ksummers/instruments/INDI/LBTI_INDI/src area.

As of 4-May-2017, we have deployed a version of the DDS subsystem that spawns an INDI thread. The Tucson web server machine runs an indiserver that talks to this DDS driver. It includes only the properties required by the FLAO web page. (A previous version also had a CamInfo property that was writable, for testing purposes.)
[web@web]$ getINDI LBTO.*.*
LBTO.Version.Driver=1
LBTO.Version.Properties=1
LBTO.Version.Uptime=82394
LBTO.Now.JD=2457879.359815904405
LBTO.Now.UTC=20.635576124166668421
....
LBTO.Events.MoonRise=21.443386819213628769
LBTO.Events.MoonSet=10.390646349638700485
....
LBTO.Weather.state=1
LBTO.Weather.Temperature=14.199999809265136719
LBTO.Weather.DewPoint=-8.3999996185302734375
LBTO.Weather.Humidity=20.100000381469726562
....
LBTO.Status.state=Ok
LBTO.Status.AzTrackingMode=TRACKER_VEL_MODE
LBTO.Status.ElTrackingMode=TRACKER_VEL_MODE
....
LBTO.AOStatus.state=Ok
LBTO.AOStatus.L_AOStatus=Ready
LBTO.AOStatus.L_AOOffsetX=-0.943112
LBTO.AOStatus.L_AOOffsetY=-7.06325
....
LBTO.Pointing.state=1
LBTO.Pointing.L_RA2K=999
LBTO.Pointing.L_Dec2K=999
LBTO.Pointing.R_RA2K=999
LBTO.Pointing.R_Dec2K=999
LBTO.Pointing.Azimuth=120.60100072700824114
LBTO.Pointing.AzVel=0.00041378987953066831288
....

The INDI port number is configurable in the DDS dds.conf file as INDIPort (currently using 7630).

Web Files

  • The indi.fcgi process is the web interface. The default Apache configuration is set up to dynamically launch the process when a web page accesses it. This causes the arguments to the process to default to localhost for its indiserver. We don't want an indiserver on the web host, so we want the indi.fcgi that our web server uses to point to our indiserver on another host. LBTI is using lighttpd which is easy to set up that way. On Apache, it's easy too, but it doesn't default.

    Add to our Apache config file (/web/server/tuc-shell64.tucson-httpd.cfg/statserv.conf) :
    ProxyPass /indi.fcgi fcgi://127.0.0.1:17624/ 

    Launch the indi.fcgi using the port specified above, on the web server:
    /usr/bin/spawn-fcgi -p 17624 /web/statserv/cgi-bin/indi.fcgi +age=2 +reconnects=100 +log=/web/server/logs/indi.fcgi.log  +host=ourindiserverhost:7624 
    This is done via the spawn-fcgi service that is already installed on web. Modified the /etc/sysconfig/spawn-fcgi file to add:
    # spawn the INDI web interface application
    OPTIONS="-p 17624 /web/statserv/cgi-bin/indi.fcgi +age=2 +reconnects=100 +log=/web/server/logs/indi.fcgi.log +host=tuc-shell64:7624" 

    Note that we have to use the mod_fcgid.c Apache module, NOT fastcgi which does not support this mode.

Command Interface to TCS

In January-2018, built the LBTI indilbto driver which allows you to command IIF.

The config file (ours is named lbto.conf) specified the name/IP of IIF and what instrument you are:
PROXY                   Factory:tcp -p 10000 -h iif.mountain.lbto.org
FOCAL_STATION           bentGregorianCenter both
INSTRUMENT_ID           LBTI 

I modified it to use my IIF IP and then I could send a preset (when LBTI was authorized) with:
indiserver -v indilbto &
setINDI "LBTO.SetStars.RA2K=20.5;Dec2K=45;Magnitude=14"
setINDI "LBTO.SetReference.RA2K=20.5;Dec2K=45;Magnitude=14"
setINDI "LBTO.Preset.Mode=TRACK;Side=both" 

OVMS Driver

Used LBTI's OVMS driver on the ovms host, chained to the indiserver on our web server.

This driver is called indiovms from the src/INDI directory. It was built from Elwood's tar file, with modifications to two Makefile files for the path to Ice on that machine and a change to find the ovms.cfg file in the LBTIINDIHOME/config directory (see notes in the /home/ovms/LBTI_INDI/ directory on the ovms host.)

It is run from the ovms host, and then run a "chained" INDI server on web as well (with the OVMS@192.168.53.60 ):
ssh ovms@ovms.mountain.lbto.org
export LBTIINDIHOME=/local/INDI
$LBTIINDIHOME/bin/runindionovms &

ssh web@web
cd /web/statserv
export LBTIINDIHOME=/web/homes/ksummers/instruments/INDI/LBTI_INDI
$LBTIINDIHOME/src/INDI/indiserver -l ./ Maths@10.130.142.176 LBTO@10.130.142.176 OVMS@192.168.53.60 &

or 

$LBTIINDIHOME/bin/runindionweb

Once the servers/drivers are running, the web page can be run to see the current values and update the sample driver
http://statserv.lbto.org/indi_ao/www/flao.html

INDI and LBC

Took Elwood's "minimal" tar file to the CMU.

To build the libraries, server, and sample driver, only had to modify the link for the two executables to include the pthread library.
INDI/Makefile and indi-sample-driver/Makefile

The minimal INDI did not have files for the API. So, when I started working on making LBC talk INDI, I had to bring over the files required.

  indidevapi.h
  indidriverbase.c   (edited to not include astro.h since it's not req'd)
  configfile.h / .c
  eventloop.h / .c 

Modified housekeeping.c to start an INDI thread that has four properties and a method to update them. However, housekeeping runs in two separate threads - one for red, one for blue. The way it's written right now, both would spawn this INDI thread...so that's not the right way. That's good, I guess, since it would only have worked for HK data anyway currently. Now, whatever we design will work for all the data because it will be a thread launched from the top-level.

LBTI's Web Graphs

There are three ways of graphing data using LBTI's web pages.

The oldest is in the www/graphs directory. A script is run that does a getINDI of data and time into a log file and calls a gnuplot script to create a png file used by the web page. This is working with a script called collectGraphData.sh for the FLAO web page.
Note that the SMT tau value is multiplied by 20 in the graphs. Why?

The other two ways require more work in the INDI driver code. The code has to create a log and set a history property in the INDI class (see LBTI code for the gate valve driver), or use a blob (see the OVMS and AO graphs for blob examples.)

Alarms Driver

The indialh driver was released operationally in April-2017: http://statserv.lbto.org
The source is in SVN in https:://svn.lbto.org/repos/indi/trunk in the directory indialh. This driver reads the .alhConfig alarm handler GUI configuration files and mines the first line of the $GUIDANCE to be the string set as an INDI property if a channel goes into an alarm state.

The page: AlarmsDriverTesting describes details of setting up the EPICS environment for testing and the non-EPICS driver used for web page checkout.

Took just a few channels from our tcs.db and ran a simple testioc:
export EPICS_HOME="/web/modules/epics"
export EPICS_BASE="${EPICS_HOME}/base-3.14.12.4"
export EPICS_HOST_ARCH=`${EPICS_BASE}/startup/EpicsHostArch`
export PATH="${PATH}:${EPICS_BASE}/bin/${EPICS_HOST_ARCH}:${EPICS_EXTENSIONS}/bin/${EPICS_HOST_ARCH}"
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${EPICS_BASE}/lib/${EPICS_HOST_ARCH}:${EPICS_HOME}/lib/${EPICS_HOST_ARCH}"

cd $EPICS_HOME/iocBoot/testioc/
./st.cmd &
 

Created a C++ INDI driver (in /home/ksummers/instruments/INDI/LBTI_IND/src/alhdriver ) that uses "EZ Channel Access". Used the LBTI common C++ framework and created alarm status properties that are modified based on an EPICS get of the STAT and SEVR fields of the specified channels.
The driver is called indialh

Originally, it got the actual values:

ALH.Version.Driver=0.1
ALH.Version.Properties=0.1
ALH.Version.Uptime=45.004045
ALH.LBC.L_DewarPres=0.045
ALH.LBC.L_DewarTemp=20.5
ALH.LBC.R_DewarPres=0.145
ALH.LBC.R_DewarTemp=
ALH.MODS.L_BlueCCDTemp=0.4
ALH.MODS.L_BlueDewarTemp=30.6
ALH.MODS.L_RedCCDTemp=0.3
ALH.MODS.L_RedDewarTemp=155.5
ALH.MODS.L_RedHEBTemp=99.5 

But, we only care about the alarm states. So, now the code is specific to the alarm states. It also checks for update and does not send out the INDI properties unless they have changed.

AO Drivers

The FLAO msgd drivers are just translators from the AO real-time DB to INDI properties.

The FLAO drivers we're using are:

flao_sx_ccd39  
flao_sx_ccd47  
flao_sx_wfs_msgd  
flao_sx_adsec_msgd
flao_dx_ccd39  
flao_dx_ccd47  
flao_dx_wfs_msgd  
flao_dx_adsec_msgd 

run on the sx/dx machines: 192.168.39.56 and 192.168.39.57


Full LBTI package from Elwood

builds its own cfitsio and zlib, pyfits, numpy

couldn't build the following:
  numpy
  pyfits
  acromagd
  indiprop
  lbtiINDIprop
  plc

The rest of it built with updates to
  • makefiles for our Ice, snmp, Qt paths, path to python to use anaconda
  • installation of fftw-3.3.5
  • a few updates for new compilers
  • deletion of old o files (why didn't they go on the clobber?)

Included in the tar ball was a web directory. The README file says it uses lighttpd and fcgi.

There are perl scripts - for checking if user has privileges, to run gnuplot, for power control, get logs, etc.

Server

[ksummers@tuc-shell64 INDI]$ ./indiserver 
Must give at least one driver
Usage: ./indiserver [options] driver [driver ...]
Purpose: server for local and remote INDI drivers
Code $Revision: 1.18 $. Protocol 1.7.
Options:
 -l d  : log messages to <d>/YYYY-MM-DD.islog, else stderr
 -m m  : kill client if gets more than this many MB behind, default 50
 -n    : ignore /tmp/noindi
 -p p  : alternate IP port, default 7624
 -v    : show key events, no traffic
 -vv   : -v + key message content
 -vvv  : -vv + complete xml
 -x    : exit after last client disconnects -- FOR PROFILING ONLY
driver : executable or device@host[:port]

[ksummers@tuc-shell64 INDI]$ ./getINDI 
Purpose: retrieve readable properties from an INDI server
$Revision: 1.7 $
Usage: ./getINDI [options] [device.property.element ...]
  Any component may be "*" to match all (beware shell metacharacters).
  Reports all properties if none specified.
  If -B then BLOBs are saved in file named device.property.element.format
  In perl try: %props = split (/[=\n]/, `getINDI`);
  Set element to one of following to return property attribute:
        _LABEL to report label
        _GROUP to report group
        _STATE to report state
         _PERM to report perm
           _TO to report timeout
           _TS to report timestamp
Output format: output is fully qualified name=value one per line
  or just value if -1 and exactly one query without wildcards.
Options:
  -1    : print just value if expecting exactly one response
  -a    : add timestamp in BLOB file name
  -B    : include fetching BLOBs
  -b    : exclude fetching BLOBs (deprecated, now the default)
  -d f  : use file descriptor f already open to server
  -f    : don't print the def* values
  -h h  : alternate host, default is localhost
  -l    : append label to each reported property (unless -1)
  -m    : keep monitoring for more updates
  -o    : send BLOBs to stdout
  -p p  : alternate port, default is 7624
  -q    : suppress some error messages
  -t t  : max time to wait, default is 2 secs; 0 is forever
  -v    : verbose (cumulative)
  -w    : show write-only properties too
Exit status:
  0: found at least one match for each query
  1: at least one query returned nothing
  2: real trouble, try repeating with -v

[ksummers@tuc-shell64 INDI]$ ./setINDI 
Purpose: set one or more writable INDI properties
$Revision: 1.12 $
   * Prototype web page: <br />
     <img src="%PUBURLPATH%/%WEB%/%TOPIC%/INDIPrototypeWebPage.png" alt="INDIPrototypeWebPage.png" width="642" height="365" />

Usage: ./setINDI [options] {[type] spec} ...

Options:
  -d f  : use file descriptor f already open to server
  -h h  : alternate host, default is localhost
  -m    : show property messages (implies -w)
  -p p  : alternate port, default is 7624
  -q    : suppress some error messages
  -t t  : max time to wait, default is 2 secs
  -v    : verbose (more are cumulative)
  -w    : wait for state to be Ok or Alert - can not be used with type flags

Spec may be either:
    device.property.e1[;e2...]=v1[;v2...]
  or
    device.property.e1=v1[;e2=v2...]
    (This form allows embedded ; in v as \;)
  If Property is a BLOB, each v is a file name

If spec is preceded by one of the following flags to indicate its type,
the transaction is much more efficient but there is no error checking:
  -x    : Text
  -n    : Number
  -s    : Switch
  -b    : BLOB

Exit status:
  0: all settings successful
  1: at least one setting was invalid
  2: real trouble, try repeating with -v

get the sample driver properties:

[ksummers@tuc-shell64 INDI]$ ./getINDI Maths.*.* Maths.Version.Driver=0.1 Maths.Version.Properties=0.1 Maths.Version.Uptime=750.14536 Maths.x.value= Maths.xmaths.abs= Maths.xmaths.sqr= Maths.xmaths.sqrt= Maths.xmaths.value= </verbatim>

Task List

  • All the references to indi.js should be the same. We have it in many places, with different references.
  • Automate the creation of the channelsfiles configuration files. Can we? had to do a lot of hand-editing to make it right; also put ECS OPS channel into IT.
  • Add monitoring of the INDI servers (OVMS too), the collectGraphData script, the indi.fcgi.
  • Does the ALH driver need to be every second?
  • What should we use for the guidance string when GUIDANCE is an http link?
  • If we don't like the order that the guidance messages are in, we could re-arrange the channels in the file. For instance, if everything in AO is in alarm and we don't like the first guidance...
  • Build an installation or procedure with everything we need to run our indiserver/drivers on another machine.
  • Can the IT driver call a script to set the guidance?
  • Should we upgrade to the indiserver 2.0? or keep using Elwood's? he says his is "far superior", but the other one supports a Python product

  • Can EPICS ALH thresholds be set dynamically? could we use INDI properties for thresholds?

  • INDI.pdf: Elwood Downey's 2007 document
Topic attachments
I Attachment Action Size Date Who Comment
INDI.pdfpdf INDI.pdf manage 59 K 09 Apr 2018 - 12:21 UnknownUser Elwood Downey's 2007 document
INDIPrototypeWebPage.pngpng INDIPrototypeWebPage.png manage 70 K 13 Dec 2016 - 19:31 UnknownUser Prototype web page
READMEEXT README manage 1 K 18 Nov 2016 - 22:29 UnknownUser INDI sample driver README file
README-LBTI-webEXT README-LBTI-web manage 3 K 16 Nov 2016 - 21:47 UnknownUser README text file from LBTI web directory
StatusServerArchitectureINDI.jpgjpg StatusServerArchitectureINDI.jpg manage 2 MB 05 May 2017 - 21:56 UnknownUser LBTO INDI status server architecture
indi-intro.pdfpdf indi-intro.pdf manage 157 K 28 Nov 2016 - 17:03 UnknownUser Introduction to INDI
indidevapi.txttxt indidevapi.txt manage 32 K 10 Mar 2017 - 22:42 UnknownUser man page for INDI device driver C-language reference API
Topic revision: r57 - 10 Dec 2018, IgorSola
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback