INDI at LBTO
LBTO is implementing INDI (
Instrument-
Neutral
Distributed
Interface) as the mechanism for instruments to provide data to the observatory and for the status server web interface.
See the generic INDI documents:
We are using code provided by the
LBTI development team. We have written several drivers for checking it out.
This page provides the details.
This page:
INDIProperties, provides the details for the properties LBTO is publishing.
LBTO INDI Architecture:
Note that there is a subtlety to designing the INDI interfaces that will be used via the web. You don't want to just put all numbers in one property, etc. The web interface creates a callback for a device.property
. So, you want to organize your properties by how they will be used in your clients.
See
LBTOINDITroubleshooting
Minimal INDI
After fighting with getting enough of the full
LBTI tar ball built that we could run some stuff, they created a "minimal INDI" tar ball that just has 3 libraries, the server code, a sample C++ driver, and a Python interface to
getINDI
.
A few makefile updates were required to set it up for LBTO's environment to use the C++ framework. It was further modified to include a few more files to use the C API.
To create a new driver, copy the
indi-sample-driver
directory and make one that does what you want.
The
pyindi.py
program can be modified to get/set/eval any INDI property. It uses the executables
getINDI
,
setINDI
,
evalINDI
. So make sure they're in your path if you want to use it.
The minimal INDI files have been captured in SVN in
https://svn.lbto.org/view/indi/trunk/ and built/released into
/lbt/tcs_devel/indi
and
/web/modules/INDI
To build/install required INDI libraries/include files:
ssh tcs@tcs-test
svn export https://svn.lbto.org/repos/indi/trunk indi
cd indi
gmake (the make will complain when it tries to install the python, but that's ok, we don't need it)
make install (copies required files to deploy subdirectory)
cd /lbt/tcs_devel/indi
rm -rf *
cp -pr ~/indi/deploy/* .
The deploy directories should also be copied to
/web/modules/INDI
for use by the
web server indiserver and drivers.
Simple Example
On the web machine in Tucson, we're currently running the
indilbtoDDS
,
indiovms
,
indialh
, and the FLAO AO drivers. To see the data published, you can do commands like:
/web/modules/INDI/bin/getINDI *.*.*
LBTO.AOStatus......
LBTO.Pointing...
LBTO.Status...
LBTO.Weather...
....
/web/modules/INDI/bin/getINDI LBTO.Weather.*
LBTO.Weather.state=1
LBTO.Weather.DewPoint=-5.0999999
LBTO.Weather.Humidity=96.900002
LBTO.Weather.Pressure=681.59998
LBTO.Weather.Rain=0
LBTO.Weather.SkyBrightness=21.059999
LBTO.Weather.Temperature=-4.6999998
LBTO.Weather.WindDir=247.59995
LBTO.Weather.WindDirF=32.599953
LBTO.Weather.WindSpeed=4.5
LBTO.Weather.WindSpeedF=7
$ getINDI ALH.*.*
ALH.Version.Driver=1.0
ALH.Version.Properties=1.0
ALH.Version.Uptime=9180.2462
ALH.AC.Guidance=OK
ALH.AC.Status=0
ALH.AO_SX.Guidance=OK
ALH.AO_SX.Status=0
ALH.AO_DX.Guidance=OK
ALH.AO_DX.Status=0
....
The following notes are about how to run the servers/drivers.
Run the server with a "driver" (this example uses the sample driver "indimaths"), logging to the local directory (dated filename.islog) - run it in the background.
ssh to tuc-shell64
export INDIHOME=/web/modules/INDI/
$INDIHOME/bin/indiserver -l ./ -vv $INDIHOME/bin/indimaths &
From another window, you can get/set properties. If you set the "x.value" property of this driver, it updates the "xmaths" properties.
$INDIHOME/bin/setINDI Maths.x.value=3
$INDIHOME/bin/getINDI Maths.*.*
Maths.Version.Driver=0.1
Maths.Version.Properties=0.1
Maths.Version.Uptime=3165.3756
Maths.x.value=3
Maths.xmaths.abs=3
Maths.xmaths.sqr=9
Maths.xmaths.sqrt=1.7320508
Maths.xmaths.value=3
When you're done, stop the server/driver:
ps -ef | grep indiserver
kill XXX (the PID associated with your indiserver)
It may be useful to play around with, watch the log file, etc.
Here's the
README file from the sample driver directory (also in
indi-sample-driver
)
LBTO Driver Example
Created another driver that gets the
params.txt
file that FACSUM and the WEATHERSTATION GUIs are reading and parses it to set some INDI parameters for us.
This driver is called
inidlbtoDDS
It can be run from the web or shell64 machines. The following example shows how to run a "chained" INDI server on web (with the device names and IP address of shell64 where our other server is running):
ssh to tuc-shell64
export INDIHOME=/home/ksummers/instruments/INDI/LBTI_INDI
$INDIHOME/src/INDI/indiserver -l ./ $INDIHOME/src/lbtoDDS/indilbtoDDS $INDIHOME/src/indi-sample-driver/indimaths &
ssh web@web
cd /web/statserv
export INDIHOME=/web/homes/ksummers/instruments/INDI/LBTI_INDI
$INDIHOME/src/INDI/indiserver -l ./ Maths@10.130.142.176 LBTO@10.130.142.176 &
From another window on either machine:
/web/modules/INDI/bin/getINDI *.*.*
/web/modules/INDI/bin/getINDI LBTO.weather.*
/web/modules/INDI/bin/getINDI LBTO.status.*
/web/modules/INDI/bin/setINDI Maths.x.value=100
Once the servers/drivers are running, the following web page can be run to see the current values and update the sample driver
http://statserv.lbto.org/ks/indi-demo.html
I have created a
TCS DDS INDI-compliant process instead of using this driver for the status server. The first test was a standalone main program that spawned an INDI thread using the port 7625. Then you can run a server that connects to it by
devicename@localhost:7625
. This test is in the
dds
directory in the
/home/ksummers/instruments/INDI/LBTI_INDI/src
area.
As of 4-May-2017, we have deployed a version of the DDS subsystem that spawns an INDI thread. The Tucson web server machine runs an indiserver that talks to this DDS driver. It includes only the properties required by the FLAO web page. (A previous version also had a
CamInfo
property that was writable, for testing purposes.)
[web@web]$ getINDI LBTO.*.*
LBTO.Version.Driver=1
LBTO.Version.Properties=1
LBTO.Version.Uptime=82394
LBTO.Now.JD=2457879.359815904405
LBTO.Now.UTC=20.635576124166668421
....
LBTO.Events.MoonRise=21.443386819213628769
LBTO.Events.MoonSet=10.390646349638700485
....
LBTO.Weather.state=1
LBTO.Weather.Temperature=14.199999809265136719
LBTO.Weather.DewPoint=-8.3999996185302734375
LBTO.Weather.Humidity=20.100000381469726562
....
LBTO.Status.state=Ok
LBTO.Status.AzTrackingMode=TRACKER_VEL_MODE
LBTO.Status.ElTrackingMode=TRACKER_VEL_MODE
....
LBTO.AOStatus.state=Ok
LBTO.AOStatus.L_AOStatus=Ready
LBTO.AOStatus.L_AOOffsetX=-0.943112
LBTO.AOStatus.L_AOOffsetY=-7.06325
....
LBTO.Pointing.state=1
LBTO.Pointing.L_RA2K=999
LBTO.Pointing.L_Dec2K=999
LBTO.Pointing.R_RA2K=999
LBTO.Pointing.R_Dec2K=999
LBTO.Pointing.Azimuth=120.60100072700824114
LBTO.Pointing.AzVel=0.00041378987953066831288
....
The INDI port number is configurable in the DDS
dds.conf
file as
INDIPort
(currently using 7630).
Web Files
- The
indi.fcgi
process is the web interface. The default Apache configuration is set up to dynamically launch the process when a web page accesses it. This causes the arguments to the process to default to localhost
for its indiserver
. We don't want an indiserver
on the web host, so we want the indi.fcgi
that our web server uses to point to our indiserver
on another host. LBTI is using lighttpd
which is easy to set up that way. On Apache, it's easy too, but it doesn't default.
Add to our Apache config file (/web/server/tuc-shell64.tucson-httpd.cfg/statserv.conf
) : ProxyPass /indi.fcgi fcgi://127.0.0.1:17624/
Launch the indi.fcgi
using the port specified above, on the web server: /usr/bin/spawn-fcgi -p 17624 /web/statserv/cgi-bin/indi.fcgi +age=2 +reconnects=100 +log=/web/server/logs/indi.fcgi.log +host=ourindiserverhost:7624
This is done via the spawn-fcgi
service that is already installed on web. Modified the /etc/sysconfig/spawn-fcgi
file to add: # spawn the INDI web interface application
OPTIONS="-p 17624 /web/statserv/cgi-bin/indi.fcgi +age=2 +reconnects=100 +log=/web/server/logs/indi.fcgi.log +host=tuc-shell64:7624"
Note that we have to use the mod_fcgid.c
Apache module, NOT fastcgi
which does not support this mode.
Command Interface to TCS
In January-2018, built the
LBTI indilbto
driver which allows you to command
IIF.
The config file (ours is named
lbto.conf
) specified the name/IP of
IIF and what instrument you are:
PROXY Factory:tcp -p 10000 -h iif.mountain.lbto.org
FOCAL_STATION bentGregorianCenter both
INSTRUMENT_ID LBTI
I modified it to use my
IIF IP and then I could send a preset (when
LBTI was authorized) with:
indiserver -v indilbto &
setINDI "LBTO.SetStars.RA2K=20.5;Dec2K=45;Magnitude=14"
setINDI "LBTO.SetReference.RA2K=20.5;Dec2K=45;Magnitude=14"
setINDI "LBTO.Preset.Mode=TRACK;Side=both"
Used
LBTI's
OVMS driver on the ovms host, chained to the indiserver on our web server.
This driver is called
indiovms
from the
src/INDI
directory. It was built from Elwood's tar file, with modifications to two
Makefile
files for the path to Ice on that machine and a change to find the
ovms.cfg
file in the
LBTIINDIHOME/config
directory (see notes in the
/home/ovms/LBTI_INDI/
directory on the ovms host.)
It is run from the ovms host, and then run a "chained" INDI server on web as well (with the
OVMS@192.168.53.60
):
ssh ovms@ovms.mountain.lbto.org
export LBTIINDIHOME=/local/INDI
$LBTIINDIHOME/bin/runindionovms &
ssh web@web
cd /web/statserv
export LBTIINDIHOME=/web/homes/ksummers/instruments/INDI/LBTI_INDI
$LBTIINDIHOME/src/INDI/indiserver -l ./ Maths@10.130.142.176 LBTO@10.130.142.176 OVMS@192.168.53.60 &
or
$LBTIINDIHOME/bin/runindionweb
Once the servers/drivers are running, the web page can be run to see the current values and update the sample driver
http://statserv.lbto.org/indi_ao/www/flao.html
INDI and LBC
Took Elwood's "minimal" tar file to the CMU.
To build the libraries, server, and sample driver, only had to modify the link for the two executables to include the pthread library.
INDI/Makefile
and
indi-sample-driver/Makefile
The minimal INDI did not have files for the API. So, when I started working on making
LBC talk INDI, I had to bring over the files required.
indidevapi.h
indidriverbase.c (edited to not include astro.h since it's not req'd)
configfile.h / .c
eventloop.h / .c
Modified
housekeeping.c
to start an INDI thread that has four properties and a method to update them. However, housekeeping runs in two separate threads - one for red, one for blue. The way it's written right now, both would spawn this INDI thread...so that's not the right way. That's good, I guess, since it would only have worked for HK data anyway currently. Now, whatever we design will work for all the data because it will be a thread launched from the top-level.
LBTI's Web Graphs
There are three ways of graphing data using
LBTI's web pages.
The oldest is in the
www/graphs
directory. A script is run that does a
getINDI
of data and time into a log file and calls a
gnuplot
script to create a
png
file used by the web page. This is working with a script called
collectGraphData.sh
for the FLAO web page.
Note that the SMT tau value is multiplied by 20 in the graphs. Why?
The other two ways require more work in the INDI driver code. The code has to create a log and set a history property in the INDI class (see
LBTI code for the gate valve driver), or use a blob (see the
OVMS and AO graphs for blob examples.)
Alarms Driver
The
indialh
driver was released operationally in April-2017:
http://statserv.lbto.org
The source is in SVN in
https:://svn.lbto.org/repos/indi/trunk in the directory
indialh
. This driver reads the
.alhConfig
alarm handler GUI configuration files and mines the first line of the
$GUIDANCE
to be the string set as an INDI property if a channel goes into an alarm state.
The page:
AlarmsDriverTesting describes details of setting up the EPICS environment for testing and the non-EPICS driver used for web page checkout.
Took just a few channels from our
tcs.db
and ran a simple testioc:
export EPICS_HOME="/web/modules/epics"
export EPICS_BASE="${EPICS_HOME}/base-3.14.12.4"
export EPICS_HOST_ARCH=`${EPICS_BASE}/startup/EpicsHostArch`
export PATH="${PATH}:${EPICS_BASE}/bin/${EPICS_HOST_ARCH}:${EPICS_EXTENSIONS}/bin/${EPICS_HOST_ARCH}"
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${EPICS_BASE}/lib/${EPICS_HOST_ARCH}:${EPICS_HOME}/lib/${EPICS_HOST_ARCH}"
cd $EPICS_HOME/iocBoot/testioc/
./st.cmd &
Created a C++ INDI driver (in
/home/ksummers/instruments/INDI/LBTI_IND/src/alhdriver
) that uses "EZ Channel Access". Used the
LBTI common C++ framework and created alarm status properties that are modified based on an EPICS get of the
STAT
and
SEVR
fields of the specified channels.
The driver is called
indialh
Originally, it got the actual values:
ALH.Version.Driver=0.1
ALH.Version.Properties=0.1
ALH.Version.Uptime=45.004045
ALH.LBC.L_DewarPres=0.045
ALH.LBC.L_DewarTemp=20.5
ALH.LBC.R_DewarPres=0.145
ALH.LBC.R_DewarTemp=
ALH.MODS.L_BlueCCDTemp=0.4
ALH.MODS.L_BlueDewarTemp=30.6
ALH.MODS.L_RedCCDTemp=0.3
ALH.MODS.L_RedDewarTemp=155.5
ALH.MODS.L_RedHEBTemp=99.5
But, we only care about the alarm states. So, now the code is specific to the alarm states. It also checks for update and does not send out the INDI properties unless they have changed.
AO Drivers
The FLAO
msgd
drivers are just translators from the AO real-time DB to INDI properties.
The FLAO drivers we're using are:
flao_sx_ccd39
flao_sx_ccd47
flao_sx_wfs_msgd
flao_sx_adsec_msgd
flao_dx_ccd39
flao_dx_ccd47
flao_dx_wfs_msgd
flao_dx_adsec_msgd
run on the sx/dx machines: 192.168.39.56 and 192.168.39.57
Full LBTI package from Elwood
builds its own cfitsio and zlib, pyfits, numpy
couldn't build the following:
numpy
pyfits
acromagd
indiprop
lbtiINDIprop
plc
The rest of it built with updates to
- makefiles for our Ice, snmp, Qt paths, path to python to use anaconda
- installation of fftw-3.3.5
- a few updates for new compilers
- deletion of old o files (why didn't they go on the clobber?)
Included in the tar ball was a
web
directory. The
README file says it uses lighttpd and fcgi.
There are perl scripts - for checking if user has privileges, to run gnuplot, for power control, get logs, etc.
Server
[ksummers@tuc-shell64 INDI]$ ./indiserver
Must give at least one driver
Usage: ./indiserver [options] driver [driver ...]
Purpose: server for local and remote INDI drivers
Code $Revision: 1.18 $. Protocol 1.7.
Options:
-l d : log messages to <d>/YYYY-MM-DD.islog, else stderr
-m m : kill client if gets more than this many MB behind, default 50
-n : ignore /tmp/noindi
-p p : alternate IP port, default 7624
-v : show key events, no traffic
-vv : -v + key message content
-vvv : -vv + complete xml
-x : exit after last client disconnects -- FOR PROFILING ONLY
driver : executable or device@host[:port]
[ksummers@tuc-shell64 INDI]$ ./getINDI
Purpose: retrieve readable properties from an INDI server
$Revision: 1.7 $
Usage: ./getINDI [options] [device.property.element ...]
Any component may be "*" to match all (beware shell metacharacters).
Reports all properties if none specified.
If -B then BLOBs are saved in file named device.property.element.format
In perl try: %props = split (/[=\n]/, `getINDI`);
Set element to one of following to return property attribute:
_LABEL to report label
_GROUP to report group
_STATE to report state
_PERM to report perm
_TO to report timeout
_TS to report timestamp
Output format: output is fully qualified name=value one per line
or just value if -1 and exactly one query without wildcards.
Options:
-1 : print just value if expecting exactly one response
-a : add timestamp in BLOB file name
-B : include fetching BLOBs
-b : exclude fetching BLOBs (deprecated, now the default)
-d f : use file descriptor f already open to server
-f : don't print the def* values
-h h : alternate host, default is localhost
-l : append label to each reported property (unless -1)
-m : keep monitoring for more updates
-o : send BLOBs to stdout
-p p : alternate port, default is 7624
-q : suppress some error messages
-t t : max time to wait, default is 2 secs; 0 is forever
-v : verbose (cumulative)
-w : show write-only properties too
Exit status:
0: found at least one match for each query
1: at least one query returned nothing
2: real trouble, try repeating with -v
[ksummers@tuc-shell64 INDI]$ ./setINDI
Purpose: set one or more writable INDI properties
$Revision: 1.12 $
* Prototype web page: <br />
<img src="%PUBURLPATH%/%WEB%/%TOPIC%/INDIPrototypeWebPage.png" alt="INDIPrototypeWebPage.png" width="642" height="365" />
Usage: ./setINDI [options] {[type] spec} ...
Options:
-d f : use file descriptor f already open to server
-h h : alternate host, default is localhost
-m : show property messages (implies -w)
-p p : alternate port, default is 7624
-q : suppress some error messages
-t t : max time to wait, default is 2 secs
-v : verbose (more are cumulative)
-w : wait for state to be Ok or Alert - can not be used with type flags
Spec may be either:
device.property.e1[;e2...]=v1[;v2...]
or
device.property.e1=v1[;e2=v2...]
(This form allows embedded ; in v as \;)
If Property is a BLOB, each v is a file name
If spec is preceded by one of the following flags to indicate its type,
the transaction is much more efficient but there is no error checking:
-x : Text
-n : Number
-s : Switch
-b : BLOB
Exit status:
0: all settings successful
1: at least one setting was invalid
2: real trouble, try repeating with -v
get the sample driver properties:
[ksummers@tuc-shell64 INDI]$ ./getINDI Maths.*.*
Maths.Version.Driver=0.1
Maths.Version.Properties=0.1
Maths.Version.Uptime=750.14536
Maths.x.value=
Maths.xmaths.abs=
Maths.xmaths.sqr=
Maths.xmaths.sqrt=
Maths.xmaths.value=
</verbatim>
Task List
- All the references to
indi.js
should be the same. We have it in many places, with different references.
- Automate the creation of the
channelsfiles
configuration files. Can we? had to do a lot of hand-editing to make it right; also put ECS OPS channel into IT.
- Add monitoring of the INDI servers (OVMS too), the
collectGraphData
script, the indi.fcgi
.
- Does the ALH driver need to be every second?
- What should we use for the guidance string when GUIDANCE is an http link?
- If we don't like the order that the guidance messages are in, we could re-arrange the channels in the file. For instance, if everything in AO is in alarm and we don't like the first guidance...
- Build an installation or procedure with everything we need to run our indiserver/drivers on another machine.
- Can the IT driver call a script to set the guidance?
- Should we upgrade to the indiserver 2.0? or keep using Elwood's? he says his is "far superior", but the other one supports a Python product
- Can EPICS ALH thresholds be set dynamically? could we use INDI properties for thresholds?