LBC Overview

LBC is the "Large binocular camera" and was the "first-light" instruments of the Large Binocular Telescope (LBT). It runs own web-based GUI and control software (mainly in PHP + C). The LBC Has two sides, Blue and Red (B and R), each optimized for the blue and red visible spectrum, and matching the prime focus of the one of the 8.4 meters mirrors of the LBC (LBC Blue is on the left side (SX) of the telescope, and LBC Red is on the right side (DX) of the telescope). Such a design encourages simultaneous multi-wavelength observation of specific targets, with both channels working at the same time to acquire and download images at different rates. The LBC Control Software is responsible for coordinating these activities by managing scientific sensors and all the ancillary devices such as rotators, filter wheels, optical correctors focusing, house-keeping information, tracking and Active Optics wavefront sensors.

Each LBC channel consists of a 1.2 meter tall and 1 meter in diameter cylinder (HUB) containing the optical corrector, the rotator, two 5-slot filter wheels and a LN2 cooled CCD cryostat. The cryostat contains four 4612x2048 pixels detectors used for scientific imaging (these are called the "science cameras") and two 256x2048 pixels frame transfer detectors used for guiding and active optics control (these are called the "technical camera"). Both groups of detectors share the same physical focal plane since technical cameras are place by side to the science cameras and are exposed by the same shutter. Such single shutter implies that guiding and active optics corrections are only available during scientific exposure and will be interrupted during readout.

Operational interface

The main interface to operate and control the LBC are web based. Point to: http://lbccontrol.mountain.lbto.org/ui/main.html in your web browser and it will load the interface as shown below.

This is a fairly well designed interface and is easy to operate. The five tabs on the left group main functions together. However note that this is a live interface and you should not operate the interface! since it will affect the LBC in real time. Usually only the telescope operators and observers should operate the LBC. During daytime, it is safe to watch the information display in the "House Keeping" tab as well as the logging display in the "Log Analyzer" tab.

The web based log analyzer is useful for some overview info. You can select the sub-log systems on top, and order the messages based on time, as well as choose to load any old log files. The display will color each log message in the window with either a red or blue color, indicating the origin of the log messages (from the red or blue channel). There are also command-line tools for log analysis intended for software developers as well. These will be described later.

In general the web interface site http://lbccontrol.mountain.lbto.org/ contains a wealth of various information, including a full explanation of the web interface.

Hardware organization

Currently there are five machines controlling the LBC system. A Linux machine called the "Central Management Unit" (CMU) coordinates all activities. There are four Windows PC that control the CCD detectors, two for each side/channel. Therefore we have "Red Technical PC" and "Red Science PC", as well as "Blue Technical PC" and "Blue Science PC". The following (outdated) diagram gives an overview of the arrangement:

This diagram is dated in that currently the Windows PC only control the CCD detector and does not do anything else. All other functionality have been migrated to the CMU. But this diagram still offers a correct view of the arrangement of the five machines. There is a plan to upgrade the Windows PC to the BeagleBone single board computers. But that hasn't happened yet (and there is currently no timeline on when will that happen).

Power Control

If you must turn on/off the LBC, the best way to do it would be to use the web interface (if it is available and active). Follow these steps (Note: normally you will not need to do this, only an observer will perform this):

First head to the web control interface http://lbccontrol.mountain.lbto.org/ui/main.html and choose the "Power Control" tab on the left:

To turn on LBC from the web interface:

• First click the "Turn On House Keeping" button.
• Then click "WarmUp" if temperature is below freezing. If this is the case, then wait until the electronics to warm up, usually up to 20 miniutes.
• Click "Turn On" on the bottom of the page.
• Finally click "Connect LBT" at the top.

To shut down the system:

• Click "Turn Off" button.
• Optionally click "Turn Off House Keeping" button if you want complete shutdown. But generally you don't need to do this as the House Keeping system stays on.
• Click "Disconnect LBT" on top.

The machines and devices are connected to Internet controlled power device. These are referred to as "Dataprobe" in most LBC documents even though it is no longer used currently. "Dataprobe" is the product name for the first generation power controller used in LBC system and therefore this name stuck and people just keep using it. The current power controller is called "iBootBar" and is functionaly equivalent.

The Windows PCs are connected to the dataprobe. To power on/off the Windows PC, either push the button on the PC, or telnet to the dataprobe unit and turn off the outlets (user name admin). The CMU is not connected to the dataprobe, you have to login to the machine to shut it down (more on that in the next section).

The following screenshot displays a telnet session to one of the dataprobe units:

You can issue many commands to the dataprobe unit (including rebooting it). Refer to the official dataprobe unit for all available commands (PDF manual in the attachments).

Another tool for troubleshooting and testing is the "power.py" script on the CMU (see next section on how to login and where to find it). This is a script that tells you the power state of each individual electronics (see a screenshot below) on both channels. You can click each individual components to cycle their power state. This tool is mainly used checking power status and troubleshooting (i.e., if LBC does not power up completely and you want to check which component got stuck and then try to fix it).

There are two notes regarding "power.py":

• The power state displayed by the tool does not automatically refresh. This is designed on purpose. This is because if you refresh too much and too often, the dataprobe unit can get stuck and cause problem. So it is best to manually refresh the status when needed. Just use the big "Refresh" button at the bottom of the GUI to manually refresh the status.
• If you only want to check the power status on a single channel (blue or red), for example, only one side is powered on, then you can use the channel command line argument to the tool (b for blue and r for red):

%CODE{lang="bash"}% $ power.py --channel b %ENDCODE%

Refer to http://lbccontrol.mountain.lbto.org/overview/network_overview.shtml for all the current network addresses for various machines and dataprobe units. Below is a diagram illustrates how all these components are wired together.

The CMU

Login

The Central Management Unit plays the role of performing most of the tasks in the LBC system. It is a Linux machine that you can log on to do things on the command-line that the web interface cannot do. To login, use the account lbccontrol:

%CODE{lang="bash"}% $ ssh lbccontrol@lbccontrol.mountain.lbto.org %ENDCODE%

You can also setup paswordless login with ssh keys. After login, you will be in the home directory of lbccontrol. The main executables and configurations are located in the root directory /lbccontrol, you can issue command "current" in and it will take you to the current version in the /lbccontrol directory.

Most Useful Programs

Following is a screenshot displaying the main executable programs you will find in the /lbccontrol directory:

There are many executables in this directory. The following list gives the most useful ones:

• lbc: This is the single executable file for the LBC system, it is written in C mainly. You don't typically execute this program, other various scripts are built to call this executable to perform certain tasks. The code also interfaces with other parts such as TCS, GUI, and the Windows CCD controller.
• lbcstart.sh and lbckill.sh: These two are command-line scripts that start and shut down the LBC control software. Usually observers use these scripts. They are also aliased as lbcstart and lbckill and both the telescope and lbccontrol users can run them.
• power.py: This is the Python script for checking and toggle power status of various electronics components. It is described in the previous section. But it is located here in the /lbccontrol/ directory.

Kellee has produced a summary of the main software programs you will find on the CMU and it was recently updated. Refer to that document (https://wiki.lbto.org/Software/LBCSoftwareDescription) for details of the many programs in the screenshot above. Especially the various test program you can run to check the status of the system.

Also note that observers typically use the IDL and IRAF tools from /home/lbcobs/supportscripts/ObserverSupport and /home/lbto/iraf/LBTtools to look at LBC data. These directories are accessible from the obsN machines.

Conf files

Configuration files live in the conf directory in the current version. For example, for version 2.15, it will be: /lbccontrol/2.15/conf. The most important configuration files are the lbc.conf for the entire system's configuration and the bluechannel.conf and redchannel.conf for configuration parameters for each individual channel. As the warning displayed inside these configuration files, you should not modify any of them unless you absolutely know what they mean. Normally there is no need to change anything in these files. And usually the science and observing staff will make such changes if needed.

Logs

Logs file live in the directory: /lbccontrol/<version>/log. The file lbc.log is the current log and previous logs are archived in the same directory according to date and a sequence number and then gzipped. The files hkblue.dat and hkred.dat are log files created by the Housekeeping subsystem. They record the temperature and pressure measured inside each cryostat.

In lbc.log, every power access does a powerdump for all five of the power devices. It represents how many users there are for each port. For example, you may find such information in the LBC log:

%CODE{lang="bash"}% powerdump: 192.168.59.113 outlets owners 20110000 last access 123 sec ago %ENDCODE%

This represents that port 1 of 59.113 (CCD controller power line) has two users (since the TRACKER and the CAMERA share the CCD controller power), port 3 has one user (Housekeeping sensors are only used by the Housekeeping subsystem), and port 4 has one user (port 4 is the shutter, only the CAMERA subsystem uses that). The housekeeping temperature and vacuum sensors share a power supply.

There are two scripts useful related to logs. They are located in /home/lbccontrol/bin directory. You can execute directly from command-line anywhere when logged in as lbccontrol since they are on the PATH variable.

• emailLogErrors.sh: this script is added to cron job that sends out emails whenever error messages are detected in the log files. It contains the email address that it will notify.
• consolidateInfo.sh: this script takes an LBC log file (gzipped or plain) and then generates an condensed version of it that only contains useful information (since the original log file may contain large amounts status information that are not particularly interesting). The script places the consolidated log file in a hidden directory .tmp in the current directory. The following screenshot shows an example of using the script.

Software Build

Consult the document https://wiki.lbto.org/Software/LBCBuildAndInstall for detailed instructions on building and releasing the software on the CMU.

Windows PC

Normally there is no need to login to the Windows PC. If needed, you will need to use the LBC KVM machine to access them remotely. Use VNC to r12-kvm.mountain.lbto.org (192.168.52.86) and then you can have single-user access to each of the windows PCs. On the KVM machine menu, you will see a list of choices, the following list gives a list of mapping between the KVM names to the actual machine:

• bsci.lbc: the Windows PC for blue science detector
• btech.lbc: the Windoes PC for the blue technical detector
• rsci.lbc: the Windows PC for the red science detector
• rtech.lbc: the Windows PC for the red technical detector
• cmu.lbc: The CMU machine

Also upon start, the Windows PC will grab the software from the folder /images, and in particular the lbc.zip file. The Windows PC has a file update.bat that is responsible for unzipping and installing the software.

Data

Data are written in the D:\ directory on the Windows PC and then ftped to the repository. Science images are coped to /lbt/data/repository/<date>. Technical chip data are coped to /lbt/data/logs/lbc/tec/[BR]Tech/<date>. Logs are in /lbt/data/logs/lbc/<year>.

Guiding

LBC uses technical chips for guiding. Images are acquired from the technical chip and processed with SeXtractor. First image output is saved as reference catalogue. Next images are cross-correlated (using SeXtractor ASSOC) with extracted sources, and distances (last - first) of 20 brightest cross-correlated stars are used for guiding.

Simple Troubleshooting

The LBC system has many individual components that are interconnected together. They have to successfully power up and communicate with each other in order for the entire system to function properly. The single most common issue when starting and operating the LBC system is that something gets stuck and does not power up or work as expected. The single most commonly used solution is just to turn on and off the entire system multiple times (and be patient). Usually most of the time, everything will eventually power up and connect, and then LBC can function properly.

Tech support reference

This section is for people who are reasonably familiar with LBC system and needs quick reference and solutions to nightly problems that might occur. To be written...

Documents

• LBC collimation
• See Software/LBCControlSoftwareNotes for description of software, machines, build procedures, troubleshooting, etc.
• Addition notes: Software/LBCSoftwareDescription.