MODS Software Handover

The ideal handover scenario includes planning, presentation, training, a build/run exercise and a support agreement with the developer.
The non-ideal handover scenario includes a lot of reverse-engineering. MODS should be close to an ideal handover - we have time to plan and be part of the commissioning and the MODS team will be around for awhile to support.

Handover Materials

  • docs are also in SVN - probably they should be deleted from SVN and put in the CAN
    • top-level doc/MODSAGW_Stage_Use_Cases_v1.3.pdf is CAN doc 603o703
    • MODSAGW_oacserver.pdf from January-2008 is not in the CAN, and is not complete
    • agwSim and agwServers both have these docs in SVN: AGWsim.pdf - AGWSim v0.2 MODS AGW Stage Simulator Agent (16-Feb-2006)
      IMPv2.pdf ICIMACS Messaging Protocol Version 2 (21-April-2004)
      MODSAGWSpec.pdf - AGW Specifications for MODS (28-April-2004)
      MODSAGWStage.pdf - MODS AGW Stage Progress Report (26-May-2004), CAN doc 603o701a

LBTO engineer should spend time going through all the existing documentation (that is current) available before the walkthrough.

Presentation

The current responsible engineer should provide a presentation to the LBTO responsible engineer that includes:

  • Software Overview
    • Requirements
    • Use Cases
    • Reference Documentation
    • Development Metrics (LOC, overall complexity, etc.)
  • Design Concepts
    • Software Architecture and Strategy
    • 3rd party products and interfaces
    • Communications Strategy
    • Languages/Scripts
    • User Interface (GUIs)
    • Performance / Design Contraints
  • Build/Run-time Concepts
    • Source Control
    • Makefiles, scripts
    • Configuration files
    • Start/Stop/Monitoring tools
    • Logging / Analysis concepts

Would be nice to walk through this info, rather than just have it presented.

Training

  • Documents Review / Feedback / Questions
  • Build Exercise
    • Source Management / Extraction
      https://svn.lbto.org/repos/mods/trunk - svn repository for the library used by GCS for MODS
      AGWSim ?
    • Build
      Ray: The AGW also works with MODS, and when we make changes, it means the changes have to be done in two places on local mods and on /trunk/mods. 'mods' builds all of mods system including the agw - GCS on a Centos 5.x computer. These libraries are built in mods computers under /home2/mods with 'make' and all the libraries are kept under /home2/mods/ulibs including the agw-GCS libraries.
      My understanding is that you build the agw-GCS code under Centos 6.x which might not be compatible with our mods.
    • Deploy/Install
  • Runtime Exercise ( accomplished in Jan-2014)
    • Start/Stop
    • Monitoring logs, errors, and performance
    • Post-run analysis (ensure performance and products are as expected)
  • Engineering Tools?
  • Test/Debug Procedures & Tools?
  • Simulation capabilities?
  • Off-Line Data Analysis Tools?

Source Code

The only thing LBTO builds is the library libagwutils.a used by GCS.

In our repository, the top-level makefile ( Makefile.build ) builds only the API and app directories - the app directory builds the libagwutils.a used by GCS.

  • The checkout in /home2/mods/trunk on the mountain is not up to date - it's got the wrong SVN address (using arizona.edu)
  • After talking with Ray about the changes for the SFPToCCD transformation changes, I cleaned up the repository: deleted bin , obj , lib , libs directories, etc. It still doesn't exactly match what's in their /home2/mods/agw_1.2.0 though. (May-2013)

Building the libagwutils.a from SVN

Log in to a 64-bit CentOS machine in Tucson (last build used tcs@tcs-test ), export/checkout to the version-number you are building, make and install.
For instance, for the last build version 1.2.3:
ssh tcs@tcs-test
cd modsagw
svn checkout https://svn.lbto.org/repos/mods/trunk/agw mods-1.2.3
cd mods-1.2.3
edit Makefile.build to set the VERSION, SUBLEVEL, PATCHLEVEL numbers for what you want (top of file)
make
make install   (copies to /lbt/tcs_devel)

ssh tcs@tcs1.mountain.lbto.org
cd /lbt/tcs_devel
scp -pr tcs@tcs-test.tucson.lbto.org:/lbt/tcs_devel/mods-1.2.3 .
  • Note that the GCS links in the static library, so the TCS file .build/environment has to be modified for the path of the latest MODS library.

Building the Instrument Control Applications from SVN

TBD

System Administration

  • The FITS files are written to newdata via a mount on the modsNdata machines in /archive/data :
    mods1data:mods% df .
    Filesystem           1K-blocks      Used Available Use% Mounted on
    mt-archive:/mnt/newdata
                         103212288   3412160  94557256   4% /archive/data

  • Identify all accounts and logins
    all the accounts on the mods1 machine use tcsh

account machine description
mods mods1
mods1data
owns the mods processes
islprog mods1 owns mods source
support mods1 owns support dir in source area
dts mods1
isluser mods1
observer mods1
LBTO
lbto
mods1
mods1data
obs
OSURC
osurc
mods1
mods1data
obs
LBTB
lbtb
mods1
mods1data
obs
INAF
inaf
mods1
mods1data
obs
AZ
az
mods1
mods1data
obs
MODSeng
modseng
obs machines support scripts

  • Identify all equipment makes, models, OS, and versions
    See: MODS Network Infrastructure Requirements v1.0 (Oct-2013)
  • Document how computers were set up and configured
  • 3rd party interfaces/equipment
  • Installed programs/packages
  • Required Licenses & Maintenance agreements
  • Periodic procedures (cron jobs) identified

Architecture

Port mapping, Switch map, Etc.

MODS has its own VLAN 192.168.139. Within that, they have assigned IPs like so:
192.168.139.0xx are common resources
192.168.139.1xx are MODS1
192.168.139.2xx are MODS2

Given a base IP address: 192.168.139.xyz and the above x values, the y is: red IEB (0) blue IEB (1) MODSx LLB and IUB (2) MODSx Instrument Computers (3) MODSx KVM console appliances (4)

MODS1 Computers
mods1 192.168.139.130
mods1data 192.168.139.131
blue ccd 1 DOS machine
red ccd1 DOS machine
MODS1 WAGOs
Instrument Utility Box (IUB) 192.168.139.122
Blue Instrument Electronic Box (IEB) 192.168.139.110
Red IEB.. 192.168.139.100
Lamp/Laser Box.. 192.168.139.120
MODS1 COMTROL Servers (for MicroLynx devices)
Blue IEB Comtrol 1 192.168.139.111
Blue IEB Comtrol 2 192.168.139.112
Blue IEB IMCS Comtrol. 192.168.139.113
Red IEB Comtrol 1 192.168.139.101
Red IEB Comtrol 192.168.139.102
Red IEB IMCS Comtrol 192.168.139.103
MODS1 AGw mods1-cam 192.168.2.35 (CRB #13, serves both cameras via azcamserver 192.168.2.199)
MODS2 AGw mods2-cam 192.168.2.36 (CRB #13, serves both cameras via azcamserver 192.168.2.199)
There's a KVM for the computers (MODS1 and MODS2)

As-built Network Architecture diagram


Internal equipment
Connectivity to backbone

how does MODS talk to the archive?
it is mounted, but it doesn't show up until you go there (because of the automount ?):
mods1data:mods% more /etc/mtab 
/dev/md1 / ext3 rw 0 0
none /proc proc rw 0 0
none /sys sysfs rw 0 0
none /dev/pts devpts rw,gid=5,mode=620 0 0
usbfs /proc/bus/usb usbfs rw 0 0
/dev/md0 /boot ext3 rw 0 0
none /dev/shm tmpfs rw 0 0
/dev/md3 /lhome ext3 rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw 0 0
automount(pid3050) /archive autofs rw,fd=5,pgrp=3050,minproto=2,maxproto=4 0 0
nfsd /proc/fs/nfsd nfsd rw 0 0

mods1data:mods% ps -ef | grep 3050
root      3050     1  0 Jul13 ?        00:00:00 /usr/sbin/automount --timeout=60 --ghost /archive file /etc/auto.archive ..... 
mods      4854  4553  0 22:48 pts/1    00:00:00 grep 3050

mods1data:mods% more /etc/auto.archive 
# $Id: MODSSoftwareHandover.txt,v 1.35 2014/01/08 19:53:54 KelleeSummers Exp $
# This is an automounter map and it has the following format
# key [ -mount-options-separated-by-comma ] location
# Details may be found in the autofs(5) manpage
#
# LBT archive machine archive1 = 192.168.38.101 [rwp/osu 2010-06-12]
# automounts archive:/newdata as /archive/data
#
data archive:/newdata

mods1data:mods% ls /archive
data/

mods1data:mods% df 
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/md1              15116744   9690760   4658084  68% /
/dev/md0                101018      9041     86761  10% /boot
none                   1037384         0   1037384   0% /dev/shm
/dev/md3             136599440  92640808  37019768  72% /lhome

mods1data:mods% cd /
/

mods1data:mods% ls -al
total 270
drwxr-xr-x   28 root root  4096 Jul 13 00:22 ./
drwxr-xr-x   28 root root  4096 Jul 13 00:22 ../
drwxr-xr-x    3 root root     0 Jul 13 00:22 archive/
-rw-r--r--    1 root root     0 Jul 13 00:22 .autofsck

mods1data:mods% cd /archive/data; pwd
/archive/data

mods1data:mods% df .
Filesystem           1K-blocks      Used Available Use% Mounted on
archive:/newdata     246076176  58100744 175475440  25% /archive/data

mods1data:mods% df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/md1              15116744   9690764   4658080  68% /
/dev/md0                101018      9041     86761  10% /boot
none                   1037384         0   1037384   0% /dev/shm
/dev/md3             136599440  92640808  37019768  72% /lhome
archive:/newdata     246076176  58100744 175475440  25% /archive/data

Spares

Strategy and vendors

They currently have a hot spare for the data server and the instrument control boxes in the rack. The boxes are not rsync 'ed, like I originally thought. They are manually updated every 6 months to 1 year.

ISorted ascending Attachment Action Size Date Who Comment
Architecture.jpgjpg Architecture.jpg manage 134 K 15 Aug 2013 - 23:11 UnknownUser MODS Software/Hardware Architecture
MODSArchitecture.JPGJPG MODSArchitecture.JPG manage 804 K 13 Aug 2013 - 21:21 UnknownUser MODS Software Architecture
MODSSoftwareHandover-I603s00201-b.pdfpdf MODSSoftwareHandover-I603s00201-b.pdf manage 796 K 25 Sep 2018 - 20:03 UnknownUser MODS Software Handover document (March-2018, version b) from VCAN
MODS_NetworkInfrastructureRequirements_v1.0.pdfpdf MODS_NetworkInfrastructureRequirements_v1.0.pdf manage 342 K 26 Nov 2013 - 18:18 UnknownUser Network Infrastructure Requirements document
MODS_NetworkInfrastructureRequirements_v1.1.pdfpdf MODS_NetworkInfrastructureRequirements_v1.1.pdf manage 324 K 19 Jun 2018 - 19:44 UnknownUser Network Infrastructure Requirements v1.1 (April-2014)
SPIE2006.pdfpdf SPIE2006.pdf manage 3 MB 07 Nov 2014 - 22:01 UnknownUser MODS for the LBT SPIE 2006 paper
MODSComputerArchitecture.pngpng MODSComputerArchitecture.png manage 83 K 29 May 2014 - 20:44 UnknownUser MOS computer architecture diagram from Network Infrastructure Requirements document
MODS_SW_HO_opkcomments.rtfrtf MODS_SW_HO_opkcomments.rtf manage 16 K 16 Apr 2018 - 20:48 UnknownUser Olga's comments on the current Software Handover Document (March-2018)
Topic revision: r58 - 25 Sep 2018, KelleeSummers
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback