Software Release Notes for Build "2014D"

Release Date: November 17, 2014
Generic Release Name: 2014D
General Description:

(executive summary from telescopework email after testing):

In this release, several important changes have occurred. Leading the list are significant alarm handler improvements for telescope mount hardware and IT host monitoring. The new alarm handler roughly doubles monitored hardware alarms, and can now directly invoke low level TCS GUIs for quickly addressing specific alarm conditions. From a telescope hardware perspective, the majority of alarms needing to be monitored are now covered. Continued improvements for instrument and software alarms will occur in the future as operators get used to the new features. New PLC logic mirroring changes to ECS subsystem changes in 2014D were also activated.

New OAControl software was exercised. This new version primarily addresses the issue of lost software baseline status, but does incorporate some better logging and minor issue corrections. The new software allows long awaited improvements for telemetry, monitoring, and error handling between GCS and AGws to proceed forward. Finally, some minor features and issues were addressed in new versions of MCSPU, FACSUM, and ENV/Weatherstation.

Telescope Operator Overview Information

• oacontrol commands can be executed from any TCS computer. They will automatically address host agw-control.

Support Astronomer Overview Information

• none

The Details

New Functionality Tests that need to be performed with the Telescope

• none

Patches to the Release

• ECSGUI Version 3.2a: Made the ALH and the ECSGUI consistent with respect to the latch pins and the overtorqued state. This state is considered only to be a Warning.
• The OSS has been patched to properly set the Dynamic Balance severity when there are no errors.
• PCS Version 3.1a: Force the HA to be in the range 0 to 11:59:59 or -11:59:59 to 0.
• PCS Version 3.1b:
  • Moved a mutex in the two routines which handle targets, setEquatorialTarget and setDifferentialRateTarget, to encompass the deletion of obsolete objects to avoid runtime conflicts which can cause the PCS to crash.
  • Modified the signature of the computeKFPCoordinates() method in the Guider class. For the GCS only the method now passes back additional information.

Known Bugs

• While NS guiding improvements continue, work remains to achieve stable NS rates above ~40 arcsecs/hr. Use of NS guiding for MODS or LUCI science is not considered ready. Of particular interest from tests last night is a curious but consistent GCS log message about failure of the MODS2 AGw probe to reach demanded X stage position (Y axis is fine). It's not clear yet whether this is an errant GCS message, or a real problem with MODS2 AGw stage motion.
• IT monitoring can sometimes experience problems with clearing alarms (some kind of Zabbix bookkeeping error). Any alarm that "sticks", may need to be disabled or the IOC restarted to clear.

Release Overview

• none

IIF list of Public Data Dictionary Variables

• IIF Public Data Dictionary Variables compatible with this release.

Operating System, System Configuration, TCS and Hardware Computers

TCS Servers / Workstations

• none

jet computer, DSP, FPGA

• none

Networks

• none

PLCs

• none

TCS Subsystems and GUIs, and significant configuration files

AGw control server

• Version 4.1
  • oacontrol is now versioned with a symbolic link to switch the version: /lbt/oacontrol/current points to the active version.
  • Improved log file (/var/log/oacontrol.log) format; no longer writes to syslog.
  • oacontrol is installed on the TCS cluster, and command line commands can be executed on any TCS computer. An environmental variable (OACONTROLHOST) contains the host computer name.
  • oacserver now does not time out when the designated AGW is not reachable; a UMAC communication failed error is returned immediately.

Azcamserver and controller computers

• none

AOS - Adaptive Optics

• AOS
  • none

• AOSGUI
  • none

DDS - Data Dictionary Server

• DDS
  • none

• DDSGUI
  • none

ECS - Enclosure Control

• ECS Version 4.1
  • The ECS thermal data has both treehouse and telescope temperatures. The treehouse temperatures have specific associated alarms. Both sets of temperatures will be in alarm if the thermal report is stale. Created separate data dictionary items for the two groups of temperatures as the treehouse temperatures can be in two possible alarm states while the telescope temperatures have one state (ecs.instrCool.TelTemperatures_severity and ecs.instrCool.THTemperatures_severity).
  • Modified the shutter door logic for setting the severity flags. The shutter door "details" information (e.g., power, direction, breaker, etc.) should not contribute to the severity flags which control navigation to the left- or right-latch station panels. Updated how the severity flags contribute to the setting of the overall shutter door severity.
  • The cable concentrator is either NORMAL or in ALARM. The severity flag is set to severity ERROR when in ALARM.
  • Fixed the logic with respect to the severity rollup for the Instrument Cooling "Details Panel". Added missing alarm checks.
  • New Events Added: ECSThermalDataStale, treehouseTemperatureAlarm, and treehouseSystemOff
  • Updated all ECS subsystem Warning or Failed events which are generated based upon host_feedback for terminology akin to the ECS internal messaging system. This is in support of the ECSGUI now using the TCS event and LSS system.
  • Completed defensive programming for the control valves (not modulating valves), dampers, and louvers in ECS subsystems: Chillers, Dampers, Dome Cooling, Mirror Ventilation, and Snow Melting. The state of these elements is indicated by two bits, so it is possible to be OPEN, CLOSED, UNKNOWN (in motion), or ILLEGAL (both OPEN and CLOSED bits latched).
  • Data dictionary item "elementState" description string updated to reflect four states: CLOSED, OPEN, UNKNOWN, and ILLEGAL.
  • New Event associated with the ILLEGAL state for the control valves. (e.g., ecs.ss.controlValve.illegalState [where ss = subsystem of ECS])
  • New DD items (and taken into account when setting severity flags) (e.g., ecs.alarm.element_IllegalState)

• ECSGUI Version 3.2
  • See http://wiki.lbto.org/bin/view/Software/SWRelBuild2014C for upgrades done to the ECSGUI Version 3.1e as patches on the 2014C for coming out of summer shutdown.
  • Modified ECSGUI code to accommodate the new states appropriately. Since the element state really consists of two items: position and condition. The user of ECSGUI will now see:

           Element Position     Element Condition
           OPEN                 Good  - green indicator
           OPEN                 Bad   - red indicator    (Formerly, Alarm image)
           CLOSED               Good  - green indicator
           CLOSED               Bad   - red indicator    (Formerly, Alarm image)
           MOVING               Good  - green indicator  (Formerly, UNDEFINED in yellow)
           UNKNOWN              Bad   - red indicator    (Formerly, Alarm image)
           ILLEGAL              Bad   - red indicator    (Formerly, Alarm image or ERROR)
           By removing the alarm images, the new system accommodates displaying actual positions even if the element is in alarm.

ENV - Environmental Sensors

• ENV Version 1.21
  • Add 'ignore' configuration variable for SMT weather which prevents both getting the weather data and writing telemetry. This is for the summer when the SMT doesn't provide weather data.
  • Add Sky Quality Monitor data acquisition. There is a flag in env.conf to enable/disable this since the SQM only supports one TCP connection. The brightness, timestamp, and 'alive' flag have been added to LBT weather telemetry.

• ENVGUI Version 2.7
  • Color SMT weather blue if in 'ignore' state.
  • Add primary and secondary swing arm temperatures to 'Temp Sensors' tab.
  • Display the Sky Quality Monitor brightness.

FACSUM - Facility Summary

• See Facility Summary (FACSUM) Release Notes

IIF - Instrument Interface

• IIF Version 8.27
  • Remove LGS telescope modes since the Preset will not specify laser guide star operation.
  • Modified/added LUCIFER and MODS data dictionary variables per Tom and Kellee.
  • Add existing non-sidereal PCS data dictionary variables to the public list per Dave T.
  • Force LBC pseudo-monocular passive preset rotation angle to 0.0 per Michele.
  • Update M1 and M2 wavefront when doing collimation (IssueTrak 5288).
  • Change the C interface to use a user specified 'proxy' name in addition to the focal station and instrument name. This allows the LBC to create as many IIF connections as it wants.

• IIFGUI Version 4.9
  • Add ARGOS status: dichroic position and laser state.

GCS - Guiding Control

• GCS_Version-2.17.5
  
  • Changed version scheme, old scheme has been removed, cosmetic changes.
  • Switched remaining DD access in all threads to use TCSAccess from GCSMem.
  • LUCI: Fixed AGw check if position is accessible before motion attempt for oacontrol.
  • Removed numbers in status detection, added GCSErrorCodes for error codes.
  • Decreased default camera temperature period checks from 15 to 30 sec.
  • Switched all DD data types BIT to BOOL.
  • Changed sequence of calls to detect end of the patrol field.
  • Corrected platescale use in CCDtoSFP for AIP.
  • Added ALH severity/error detection when AGw and Cameras are not available.
  • Added DD name probeMovensLock.
  • Added ~20 DD names, and ~10 event names.
  • Added warnings for end of the patrol field and end of NS guiding.
  • Correcred severity for AGWUnit and added error codes to match oacontrol.
  • Removed reverences to unused DD name integrationTime.
  • Changed probe motion pattern based on accumulated error.
  • Removed periodic 1.5" "sawtooth" guide corrections reported by JH.
  • Added proactive probe motion.
  • Added NS probe config parameter prb_step_coeff, recommended value 1.0 for 30-60 arcs/h 1.6-1.8 for 120 arcs/h
• GCSGUI Ver-0.1.35
  
  • Switched DD access to use TCSAccess from GCSMem.
  • Added GuideCam exposure status.
  • Added WFScam exposure status.
  • Fixed WFScam loop timers and GUI.
  • Added GuideProbe motion status for NS.
  • Fixed GuideProbe motion time.
  • Switched DD access to gcs.side[side].
  • Fixed zernikes panel with new DD access.
  • Removed reverences to unused DD name integrationTime.
  • Enabled Probe motion and Camera exposures activity statuses.

LSS - Logging and Events

• LSS
  • none

• LSSGUI
  • none

MCS / mcstemp - Mount Control

• MCS
  • About 260 severity flags have been added to the Data Dictionary so that they can be referenced by the Alarm Handler software. The portions of AZDrive, ELDrive, DriveCabinet, HydrostaticBearing, Rotators, and Enclosure class which read status reports from the MCSPU have been altered to also maintain the alarm handler severity flags. A few very minor adjustments have been made to a few of the MCS GUIs to bring the displays into agreement with the Alarm Handler. The communication with the MCSPU has been checked in simulation mode and the Alarm Handler seems to be reading the severity flags correctly. No changes were made to any control or logic portions of the MCS, so control problems are not anticipated.

• MCSGUI
  • There have been a couple of very small changes made to a few of the GUIs. The Azimuth GUI had TPLC-heartbeat added to it. Elevation GUI had a "Servo hardware" annunciator added to it. The rotator status GUI had the "3-Phase Power" annunciator changed to "POK". Tool tips provide further explanation in all cases.

OSS - Optical Support

• OSS Version 3.7
  • Populate dynamic balance basic status: estop, stowed at Zenith, enabled, stow pins enabled.
  • If estop, severity is error.
  • Update pmac program. Improve error messages, and add same Umac retry mechanism for timeouts as subsystem uses.
  • Prevent M2 and M3 init, deinit, and home if motion is already in progress.
  • Do deinit and startup if M3 init fails so UMAC state is known.
  • Do not check selector brake in M3 init because it will be set correctly immediately.
  • Require M3 to be initialized for home operations.
  • Clear mirror home in M3 init.
  • Clear mirror and selector home in M3 deinit.

• OSSGUI Version 3.6
  • Add displays for dynamic balance basic status: estop, stowed at Zenith, enabled.
  • Delete Events object in MainWindow destructor so LSS callback is unregistered.
  • Alarm handler functionality: support command line argument to request specific sub form, and then hide main window.

PCS - Pointing Control

• PCS Version 3.1b
  • Moved a mutex in the two routines which handle targets, setEquatorialTarget and setDifferentialRateTarget, to encompass the deletion of obsolete objects. The left and right threads of a failed synchronized Preset both attempted to delete the object at the same time, causing the PCS to crash.
  • Modified the signature of the computeKFPCoordinates() method in the Guider class. For the GCS only the method now passes back the following for the provided guide star:
    • x-position in sfp (mm) double
    • y-position in spf (mm) double
    • x-velocity in sfp (mm/hr) double
    • y-velocity in spf (mm/hr) double
    • seconds since the epoch (s) double
    • delta_seconds since the epoch (s) double - where delta_seconds since the epoch is the difference between the current computation and the previous computation. The value of delta_seconds is zero for the very first computation after a PCS restart (first request by GCS for the KFP coordinates).
    • mjdutc (s)

• PCS Version 3.1a
  • Force the HA to be in the range 0 to 11:59:59 or -11:59:59 to 0.

• PCS Version 3.1
  • Modified SYSLOG text in SetCentroid routine of the Guider class for use by J.Hill's diagnostic plots.
  • A problem detected with LBC guiding in pseudo-monocular mode was a by-product of performing offsets in DETXY space. As such, this LBC only problem would be present in ANY telescope mode where a DETXY offset was done with LBC. Since LBC in standard observing does not use DETXY offsets, this problem has not been seen in nominal LBC binocular observations. The PCS initially implemented DETXY offsets for LBC in TCS Build 2013D. This functionality was tested and worked properly. However, a modification was done to this code in TCS Build 2014B to ensure the initial LBC guide update of (0.0, 0.0) (issued at the beginning of every new exposure) retained its characteristic of absolute zero. The pointing origin (PO) should be used in all computations for the guide update when a DETXY offset has been done or a pointing origin different than (0.0, 0.0) has been introduced. In my not-so-clever fix to force absolute zero values for the initial guide update, I broke LBC guiding for DETXY offsets and PO changes.

• PCSGUI
  • none

Pointing Kernel / Pointing Kernel tools

• See Software 2014C Release Notes

PMC - Primary Mirror Cell

• PMC Version 5.32
  • Read various limits from pmcLimits.conf instead of reflective memory. This allows limits to be changed without needing a new build.
  • Set thermalDataOK false and time_severity to error if the thermal computer is not reachable. This makes the GUI show red in the right places.
  • Protect all VxInterface operations with mutexes to avoid VxWorks communication confusion leading to 'socket select timeout' errors.

• PMCGUI Version 6.8
  • Add configuration status displays (for base_severity) to actform and thermalform.
  • Read various limits from pmcLimits.conf instead of reflective memory. This allows limits to be changed without needing a new build.
  • Alarm handler functionality: support command line argument to request specific sub form, and then hide main window.

PSF - Point Spread Function

• PSF Version 6.8
  • Clear base_severity flags in setNewInstrument, and only set them if the input configuration is bad.
  • "Cascade" Z4 compensation from Z11 and Z22 terms generated from crosstalk correction.
  • Properly overlap primary and secondary mirror motion in PSF::applyZernikes.
  • Change PSF main loop primary mirror wavefront update period from 10 to 90 seconds (IssueTrak 5212).
  • Change focusComp variable from int to bool.
  • Save/restore Z11/Z22 focus compensation setting when changing split/primary/secondary setting.
  • Remove relZernikes option since it is no longer viable.
  • Clean up primary mirror initMirrorWavefront and add PMWAVEDATFILEINVALID error.
  • When the temperature is bad, use the last known good value instead of zero when doing collimation and wavefront temperature compensation (IssueTrak 5283).

• PSFGUI Version 6.7
  • Display configuration status (base_severity) errors.
  • Read various limits from psfLimits.conf instead of reflective memory. This allows limits to be changed without needing a new build.
  • Alarm handler functionality: support command line argument to request specific sub form, and then hide main window.

TCSGUI

• TCSGUI Version 2.6
  • Remove the temporary disk file used to store status so ownership issues do not arise.
  • Disable start server buttons on mountain except for specified user (default 'telescope').

tcs.conf

• Remove the requirements that PCS must run on tcs2, and GCSL and GCLR must run on tcsc4.
• Add obs6 to server list on mountain.

WEATHERSTATION - Weather Station

• See Weather Station Release Notes

TCS Common Software

commandreturn - client command status returns

• none

commandsequencer - client/subsystem command handling

• none

core - foundation routines

• Add static method System::pid() to return the pid of a named process.
• Add method Shm::zero() to zero a shared memory segment.

datadictionary - reflective memory and event support

• none

eventhandler - event handling

• none

networkconfig - subsystem management

• Add option 'top' to netconfig to invoke top for just the TCS processes.
• Add option 'remove' to netconfig to remove the networkserver shared memory segment. Also remove the 'procs.conf' file in /var/tmp.
• Add options 'zero' and 'ZERO' to zero the networkserver shared memory segment. Also remove the 'procs.conf' file in /var/tmp.
• Require particular user (configurable, default 'telescope') on mountain to start network servers.

reflectivememory - global shared memory

• Use new Shm::zero() method to zero the reflective memory in Gshm::zero().

rpc - communication protocol

• none

setvalue - reflective memory access

• none

tcs_config - tcs configuration

• none

TCS Tools

DDEditor - Data Dictionary Editor

• none

ddeditor - Data Dictionary Editor

• none

DDBrowser - Data Dictionary Browser

• none

DDViewer - Data Dictionary Viewer

• none

EventEditor - Event Editor

• none

eventeditor - Event Editor

• none

Telemetry

Collector

• Telemetry release r19.
  • Add error checking for all attribute create, write, and close operations.
  • Add error checking for all other close operations. These are both in response to IssueTrak 5306 and do not change the collection interface. The new library is installed both downtown and on the mountain, and the symbolic links changed. When a subsystem is restarted it will use the new library.
  • Make the "giving up on sending telemetry" message a critical callback instead of a notice callback.

Instruments and Environmental Sensors

DIMM - Differential Image Motion Monitor

• none

IRTC - Infrared Test Camera

• IRTC Version 4.2
  • IRC
    • Change default length of non-sidereal ephemeris from 13 to 15 hours at request of John Hill.

• • NSIGUI
    • Change default length of non-sidereal ephemeris from 13 to 15 hours at request of John Hill.

LBC

• none

LUCIFER

• none

LBTO Weather Station

• none

MAT - Mount Axilliary Telescope

• none

OVMS - OPD Vibration Monitoring System

All Sky Monitor

• none

-- %USERSIG{MicheleDeLaPena - 2014-09-11}%

Comments