Please note that the following notes apply to the MySQL database version of telemetry collection.
This page does not apply to any telemetry data collected since the end of January-2013.

Extracting All DB Into Text Files

Before deleting the entire MySQL db, we extracted some telemetry into text files (Summer-2014). We extracted just 14GB of the 9TB database.

The telextract script allows you to use the stored procedures to get telemeters based on an epoch time.
The db was resurrected on tel1.tucson.lbto.org with MySQL from /tel-db
When I set the HOST correctly in the script for the tel1.tucson.lbto.org , it worked. You have to use the script to get all the different instantiations of the streams. Each has a different epoch that has to be used when actually extracting the data.

I used telextract to get all of the epochs/stream names. Then, I built a script that combined these into arguments to the telextract to extract the actual telemetry like this:

./telextract    '/wxtc/smt weather' from        4800852935.983195        >  wxtc-smt-weather-4800852935.txt
./telextract    '/wxtc/anemometer' from         4800852953.301959        >  wxtc-anemometer-4800852953.txt

Some streams would not extract like that because the name hierarchy tables were incorrect in the DB. Sometimes you get NULL from the get_system procedure. This causes the telextract script to fail. So, I created another option to telextract that takes the stream ID to dump the measures. I got this info by changing the dump of streams to include the stream ID. To use this option, put the stream ID as the first argument and use the keyword at after it.

./telextract 69540       at     4843254615.478408        >  gcs-sx-guiding-4843254615.txt
./telextract 69787       at     4843338147.261582        >  gcs-sx-guiding-4843338147.txt

The extracted files are currently on the tel-old SAN disk mounted on tel1.tucson.lbto.org available from https://telemetry.lbto.org/data/old/

disk:/FILESYSTEMS/tel-old
                      83844096  29844480  53999616  36% /lbt/telemetry_data/old 

The files are named with a timestamp in "seconds TAI" that is the epoch time for that stream. To find the data you are interested in, you need to know the TAI timeframe.

The directory structure in /tel-db/extraction :

drwxr-xr-x 2 teladmin users  20480 Jul 24 17:44 ecs
dr-xr-xr-x 6 teladmin users    50 Jul 29 15:28 AllSky
dr-xr-xr-x 8 teladmin users 12288 Jul 28 21:08 ecs
dr-xr-xr-x 4 teladmin users    28 Sep 17 21:14 gcsl
dr-xr-xr-x 2 teladmin users    37 Jul 23 20:01 iif
dr-xr-xr-x 8 teladmin users 45056 Jul 28 21:08 pcs
dr-xr-xr-x 7 teladmin users  8192 Sep 18 14:59 psf
dr-xr-xr-x 6 teladmin users 12288 Jul 28 21:08 wxtc

Each file is tab-delimited, with a header that lists the fields following.

My notes and scripts, etc. are in /home/ksummers/telemetry/tel-db/extraction

I used https://svn.lbto.org/view/telemetry/branches/r17/store/ to get telextract and updated that version with my changes.

User Access

• Telemetry Access - links to the web interface, notes on troubleshooting, etc.
• Telemetry CLI (script interface)

Database Procedures

• Data Maintenance Procedures
• Database Recovery Procedure
• How to switch tel-test between test instance and restore instance
• Restoring Database from Tape
• Database Troubleshooting

Runtime Database Configuration

• Current Configuration
• Desired Configuration
• Runtime Dependencies
• Development Dependences

Other Telemetry Wiki Docs

• Standalone Exporter - details for telemeter source developer testing with a standalone exporter
• Telemetry Controller - details for how the requirements were gathered: the process, audio files of interviews and more