Prev Next

SW&IT Meeting Minutes 2018-03-19

Attendees: Andrew, David, Doug, Igor, Kellee, Matthieu, Petr, Stephen, Xianyu, Yang

Previous Action items


Issue Number Status Next Action Priority Subject Closed On Last Activity By Solution Last Activity On Assigned To
6138 Open Pramskiy, Alexander Medium LUCI stops executing a script   Rothberg, barry   3/17/18 6:34 Admin, Software
7146 Open Hooper, Stephen High Mountain Synology Passive server problems   Hooper, Stephen   3/16/18 11:45 Admin, Software
7150 Open Edmeades, David Medium Local VM storage install   Bec, Matthieu   3/15/18 18:48 Admin, Software
7141 Open Zhang, Xianyu Low AOS stopAO command triggers Ice exceptions, transient alarms   Bec, Matthieu   3/15/18 16:27 Admin, Software
7145 Open Mason, Jerry Medium some MODS headers have duplicate keywords   Reynolds, Robert   3/15/18 16:02 Admin, Instrument
6831 Closed Admin, Software High Spurious Co-Pointing Limit Error with PEPSIPOL 3/15/18 14:40 Kubanek, Petr Included in 2017A December patch and 2018A. Please reopen if reoccurs. 3/15/18 14:40 Admin, Software
7084 Open Kubanek, Petr High DUT1 value shall be updated from IRS bulletins   Kubanek, Petr   3/15/18 14:35 Admin, Software
7127 Open   High Inconsistency in LUCI MOS configuration files   Bec, Matthieu   3/14/18 15:16 Admin, Instrument
7151 Open   Medium Repeated DX ARGOS BCU crashes.   Cardwell, Andrew   3/14/18 14:32 Admin, Engineering
7134 Closed Edmeades, David Medium Synology Rackstation1 failure, March 2018 3/14/18 12:21 Edmeades, David The data have finished replicating and the system claims to be in full HA mode at this time. My impression is that the replication was completed faster than in previous iterations, but that is not backed with data. As discussed in the shutdown meeting, I plan to upgrade the DSM to latest on a day when it is most favorable to announce a half-day outage of LBTO services. 3/14/18 12:21 Admin, Software
7144 Open Sola, Igor Medium LBC Sequence Iterator Filter order not preserved   Sola, Igor   3/14/18 9:08 Admin, Software
6959 Open   Medium AO: Internal WFS Arbitrator Loop stops when rotator angle is > 450.0   Miller, Doug   3/13/18 16:43 Admin, IQ
7061 Open   Medium Fastlink connections and configuration tracking   Rahmer, Gustavo   3/13/18 13:29 Admin, IQ
5357 Open Summers, Kellee Medium Please add the HBS oil temperature to the alarm handler 4/28/15 9:05 Kubanek, Petr Added to 2015B. A warning for a 5 deg difference between the return oil temp (TT0423) and the setpoint occurs after five minutes; an alarm occurs for a delta of 10 degrees. These alarms occur only in when the controller is running in automatic mode. If the temperature cannot be controlled (either due to the relays or PLC logic), the controller can be placed in manual mode and the Belimo valve manually adjusted. Additionally, another warning was added to ensure one of the two HBS recirculation pumps is running. In all cases, the alarms are not latched, requiring no reset operations. 3/12/18 16:52 Admin, Software
7143 Open Sola, Igor Medium MODS ROI command for imaging   Sola, Igor   3/12/18 11:58 Admin, Software
7029 Open Hooper, Stephen Medium rm507 monitors fail to wake   Power, Jennifer   3/12/18 0:44 Admin, Software

Team reports


  • ARGOS pointing
    • Discussed possible solutions using the LAT with DS
  • MASS
    • Guider to be repaired in Tucson
  • ARGOS "lite" status GUI
    • Request from Jennifer Power for LUCI operators
    • Display arbitrator status and satellite avoidance info instead of the full blown status GUI
  • ARGOS alarming and telemetry
    • Started to think of the instrument interface to TCS, etc.


  • Concatenation of two short weeks
    • Synology, continued
      • Received replacement unit
      • Wrong firmware, cannot installDSM 5.x to link HA pair
      • Synology charged us for the old unit the day after I received the replacement, counter to all expectations of how an advance RMA works. Louise is working on it.
        • Old unit sent out on 6 March with assurances that the charge would be canceled.
        • This turned out to be true; credit card is happy again.
      • Working on Synology's instructions to prepare the unit for a level 2 tech to install DSM 5.2
        • Teamviewer connection worked; all future support should use this method for security
        • The "special modifications" ended up being editing two text config files
          • /etc.defaults/VERSION
            • majorversion="5"
          • Add a line to /etc.defaults/synoinfo.conf
            • skip_codesign="yes"
          • After those, the external Synology Manager app will be able to install DSM 5.2
      • Despite some speedbumps detailed in IT 7134, I was able to reestablish the HA pair and begin data replication.
        • I believe the problem to be due to miscabling of the backplane, but I can't confirm that without taking live gear down. We can add this item to the DSM upgrade downtime if desired.
    • Ordered SSDs and caddies to create local VM storage on VM hosts
      • Drives installed in caddies; basic procedure proposal for install outlined in IT 7150
    • Received Laura's new laptop
      • OS debloat, software install
      • Had to RMA monitor damaged in shipping
    • VM work
      • Rename/renumber Fedora VM for SWG dev use
      • Clone VM for Xianyu
    • Mountain Synology issue
      • Ran scan on drive pulled from mountain array; no I/O or data corruption problems found.
    • Info gathtering meetings with Dell/EMC regarding Synology replacement
      • Unity platform
      • IMO it's the same broad market segment as Synology, but designed with enterprise clients in mind
      • I'll create a document that compares the two products


  • Transition activities
  • Acquisition Camera
    • Discussion with Engineering & ARGOS SW (Jonathan, Andrew)
  • Pointing Analysis (cont)
  • Minor assistance for past SW interview candidate to Tom M. - ISP candidate now
  • This week:
    • Acquisition Camera
    • Pointing
      • MODS
      • ARGOS "work-around"
    • Collimation performance
    • In Tucson (Friday)



Last week
  • Actuator 213
    • Preliminary investigation with PK and Mike Gardiner
    • Many iterations of raising + lowering the mirror to enable testing
    • Could not test actuator without PMC panic.
  • Planner
    • Demo for MB
    • Scaling feature added to improve step sequence readability.
    • DS emailed ideas for tests that involve copointing
  • Computers
    • jastier-ws running fine with 16 GB RAM
    • jastier-dev has 8 GB, insufficient (SH notified)
This week
  • Planner
    • Incorporate DS ideas into our test plan and give it a try
  • Issue Traks
    • 6799 (cut and paste loses info)
    • 7074 (ARGOS GS valid modes)


  • Made good progress on the OAC HK upgrade. The set alarms now works through the command-line interface.
    Parsing the alarm string and putting it into the getstatus.
    Haven't heard back from Dan and Mike about what else they need to be able to test in the lab in Tucson - but I think this is all they should require.
  • Summer shutdown research and discussions.

This week
  • Make sure the error-handling is correct in OAC - closing the socket each time, etc.
  • Work on what to do in OAC with HK alarms.
  • Prepare for Tucson trip - make sure I have all the LBC dataprobe info, who else to meet with, etc.


  • management (OC, WPC, AO with JC), meetings (Argos, Soul, see calendar), daily IssueTrack follow up
  • shutdown plan meeting, circulated summary for comments
  • iteration and prototype git, rpm, Jenkins CI, rpm, yum serve developments
  • svn survey, feedback requested
  • responsibilities and assignments draft
  • techtalk#1 git//ci/rpm/kickstart
  • issuetrack api
  • github triggers, build server kickstart


  • IT 7084 - DUT1 - keep exchanging ideas, added telemetry with DUT1 values
  • svngit wiki
  • spend most of Thursday and Friday dealing with DX PMC (actuator 233 problem)
    • turned to be 213 problem
    • finds out by disabling all actuators on board 0 (see T481s041, page 167, for assignment of actuators to boards and addresses) and the re-enabling actuators; problem started after we hit 213
    • there was not fully understandable message in PMC which halted our progress for a while (return value was not masked for address part, so instead of receiving 8 which will point to 213 by address, we got 72, 200 - all masked with 0x1f gives 8) - fixed
    • theory: 213 control board starts to reply to other packages after being addressed. Can be HW fault or SW fault - we can reload firmware (on the telescope) next time we see this issue. See VxWorks for details.




Action items

  • SWG Summer Shutdown 2018 plan - follow up 'need more info' items, OC / Laura engineering plan (MB)
  • swg management - propose new tools (MB)
  • github watch (All)
  • svn survey (All)
  • wiki report guideline, privacy ? (MB)

-- %USERSIG{MatthieuBec}%
Topic revision: r19 - 11 Dec 2018, PetrKubanek
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback