You are here: Foswiki>Commissioning Web>Com20081204 (09 Oct 2009, NormCushing)Edit Attach

Telescope/TCS Checkout for LUCIFER Com2
UT 2008 December 04

Observers: D.Thompson (@LBT) with N.Ageorges, W.Seifert, P.Buschkamp, M.Juette, V.Naranjo, several others from LUCIFER team. J.Hill in Tucson until 11 PM.
Telescope Operator: D.Gonzalez-Huerta
SW Support: N.Cushing, M.De La Pena, T.Leibold in Tucson until ~midnight

Data will eventually be copied to /Repository/LUCIFER_Data/20081203/

Summary:

  • Rocky startup, multiple (8-10?) HBS failures throughout the night. See Issuetracks 1874, 1876 and 988.
  • Failed to focus LUCIFER as the seeing was too variable (also seen on Polaris mon).
  • Updated LEFTZEROPOINT to -187.455 (should actually be -187.448)
  • Switched back to LUCIFERSX.20081017.ptmod (was running irtc model at start)
  • Failed to align AGW - inconsistent results (0.15-0.38 degree rotation in 3 measurements)
  • Obtained 20+ collimation points from 83-20 degrees.
  • No pointing model observations - no time.

Odd WFS images at 9:53 traced to the fact that we were facing away from the wind with the vent doors closed. As soon as we opened the vent doors, the WFS settled down dramatically.

Details:

During startup: Implicit Build16 TCS checkout tests completed: TCS services, subsystems, and GUIs launched properly.

04:25 Verified clocks on LUCIFER, AGW, TCS computers are in sync. All computers checked out to be within 1ms of the GPS server on the mountain.

04:40MST DAshby starting balance run after reconfiguration from LBC to LUCIFER. HBS problem encountered. Balance completed 05:51. HBS running in "LOCAL/AUTO" and system is running, but no clear solution found. Pump #4 is off.

05:51 Fill LBC Blue
06:09 Reset encoders...06:17 HBS failed again. 06:25 HBS failed again, but problem identified on Win_CC?

Reset encoders completed 06:45.

06:50 lbtmu01 has crashed, does not respond to a ping from lbtmu05. Michele says it was a network problem and not a crashed machine.

06:53 logged into agw-control as agw-user. A startAGW completed as expected, but motor homing returns RPC errors. Torsten is checking. Apparently oacserver/check4nports were run incorrectly.

07:30 preset.sh script run...completed without problems. It failed as telescope on lbtmu01 but worked as LBTO on lbtmu05. The config file(s) in the repsective /home directories are different for LBTO and telescope.

07:33 Go to zenith for Peter to make some change on LUCIFER.

07:53 Dome now open. At zenith to switch LUCIFER masks. Took quick set of data for flexure compensation measurement.

08:02 Removing mask, switch to imaging with LUCIFER.

08:09 Preset to BS9101_GS03 from LUCIFER. Preset completed with no complaints. Adjusted from IE=-4, CA=-69 at start, to IE=88, CA=-73.

08:22 Preset in ACTIVE mode to collimate. Failed, star not found (star is in acquire000001 at 3412 DN with a background of 1280 DN - should have been found). GCS died ~08:29. My (DJT) fault, we removed an Xflip in the PCS transformation for LUCIFER that should have remained. This keeps the PCS_XY coordinate systems the same between LUCIFER and the IRTC.

08:52 Preset in ACTIVE mode to collimate. Star now found. WFS not getting data, restarting AGW and re-homing motors.

Preset in ACTIVE mode again, same source. 09:04 first WFS image. Converged to under 200nm in 4-5 cycles. Guider is showing 0.45" FWHM in short exposures.

09:12 Preset to BS9105 ACTIVE using off-axis guide star (EL=61 at start). Preset failed, no guide star found. Updating to IE=88, CA=-79.

09:19 Preset again, failed, no star.

09:24 Preset to ACT0227, pointing looks OK to ~1 arcszec or so. Checking LUCIFER rotator center coordinates...OK. BS9105 preset failed because the "star" is off-center on an edge-on galaxy!

09:35 Preset to BS9107, off-axis guide star...GCS/AGW/AZcam having problems. Rebooting the agw-control computer, homing motors.

09:53 Preset to BS9107, off-axis guide star...succeeded. WFS images look odd, and the system is not converging well (frequently getting 800-1000nm rms).

We were facing away from the wind, with the vent doors closed and the oil getting to the pads at 11C. The conjecture is that the WFS convergence problems were just "local" seeing from the heat load in the dome. When we opened the vent doors the seeing immediately improved by a factor of 2 and the WFS converged to below 200nm in one or two cycles. BS9107 is transiting in 15 minutes, changing stars.

10:44 Preset to 9108, off-axis guide star.

10:53 Start focus sequence. AIP_L.cfg shows focus=31.6mm
  • Data: luci_20081203_
    LUCI# Focus FWHM FWHMx FWHMy
    0005 28.676 6.73 8.1 5.0
    0006 22.676 4.84 5.4 4.2
    0007 25.676 6.96 7.4 6.5
    0008 28.676 6.99 7.7 6.2
    0009 31.676 7.44 8.5 6.2
    0010 34.676 9.84 11.1 8.4
    0011 28.676 7.32 7.9 6.7
  • Results: Data are not parabolic.

11:33 Start focus sequence. AIP_L.cfg shows focus=31.6mm. This time, we are stopping the WFS during the time we are integrating on LUCIFER. LUCIFER is taking 15x2.0s in K.
  • Data: luci_20081203_
    LUCI# Focus FWHM FWHMx FWHMy
    0012 28.676 6.05 6.8 5.2
    0013 22.676 5.66 5.9 5.4
    0014 25.676 6.98 7.6 6.3

11:48 HBS Faulted. Seeing appears to be varying by enough that we will not be able to measure the focus.

11:58 Preset to AF_PA190 field for astrometric measurement. Preset failed...no guide star!

12:02 Preset to WT10_282 to check/correct pointing. We are off...setting IE=79.6, CA=-58.8 (~25 arcsec offset).

12:06 Preset to AF_PA190 field for astrometric measurement.
  • Data: luci_20081203_
    LUCI# Field/Description PA(requested) PA(WCSfit)
    0015 AF_PA190 190.0 190.0012
    0016 offset 15 arcsec    
    0017 HBS faulted    
  • Results: On 0015 I get XYscales of 0.11986/0.11977 and Yrot=190.0012 degrees, although sky subtraction was not good (used 0014 from earlier focus run).

Offset pointing in the dither script was failing at the IRS (see 00:13:41 in the log on 105), but a manual offset worked, both from the LUCIFER interface. LUCIFER was authorized on left, and LBC on right, but LUCIFER is apparently only sending left requests.

Preset failed "no guide star". See acquire000016: star is 4000 counts above background but at the right edge of the acquisition image.

Astrometric solutions on Nancy's field:

N Field PAreq Xmag Ymag PAachieved
0015 AF_PA190 190.0 0.119856 0.119766 190.0012
0016 AF_PA190 190.0 0.120122 0.120096 189.9460
0018 AF_PA190 190.0 0.120070 0.120035 190.0104
0019 AF_PA190 190.0 0.119836 0.119910 190.0997
0020 AF_PA190 190.0 0.120042 0.120036 189.9963
0021 AF_PA190 190.0 0.120009 0.119864 189.9788
0022 AF_PA190 190.0 0.120075 0.119721 190.0223
0037 AF_PA-100 260.0 0.119996 0.119915 260.0211
0038 AF_PA-40 320.0 0.119973 0.120162 320.0344
0040 AF_PA100 100.0 0.119947 0.119984 99.9998
0041 AF_PA100 100.0 0.120123 0.120076 100.0525

Yscale mean=0.1199604545 sigma=1.39007E-4 Xscale mean=0.1200044545 sigma=9.66182E-5

LEFTZEROPOINT changed to -187.455 in PCSInstrument.conf file (changed to -187.448 with more data analysis).
LEFTPOINTINGMODEL changed to LUCIFERSX.20081017.ptmod (IE=-14.91 CA=-10.65)
We were running on the irtc pointing model, which would explain the failed presets due to no guide star.

Checked pointing on WT10_282: updated IE=90.7, CA=0.0. The large offset in IE is consistent with the large offset applied to the irtc pointing model that was in effect.

0042 AF_PA190 190.0 0.120090 0.119814 190.0093
0043 AF_PA190 190.0 0.120089 0.120189 190.0201
0043 max=20 190.0 0.119942 0.120141 190.0092

We should be aligned to better than 0.02 degrees on LUCIFER.

02:05 Correct AGW-PCS Transformation.

46-48 0.148 degrees (offset in DET_X) 49-51 0.373 degrees (offset in DET_Y) 54-55 0.283 degrees (offset in DET_Y) 56-57 HBS failed during 0057

02:42 HBS problems again (same error). Back up but immediately failed again on preset.

03:30 BS9134 The Polaris monitor is reading around 2 arcsec FWHM.

03:38 HBS failed again. Nancy says the HBS crashes for her have been occurring regularly on the first large-ish (~3 arcmin) offset in a script. Waking John Little.

There is clearly something not understood about the AGW transformation, and we are running out of time, so we are going on to try to take collimation data.

04:40 Take collimation data:
  • Data:
MST LUCI X Y Z RX RY RZ EL Temp Source
04:46 0059 -0.300 -0.636 -0.293 -19.323 0.327 0.000 55.5 0.30 C08+00
04:51 0060 -0.785 -0.304 -0.077 -17.833 2.989 0.000 65.9 0.30 C08+09
05:00 0061 -1.007 -0.202 -0.103 -17.077 5.193 0.000 66.9 0.28 C08+20
05:06 0062 -0.990 0.041 -0.178 -15.107 4.733 0.000 68.5 0.26 C08+28
05:12 0063 -1.442 0.573 -0.782 -18.120 7.822 0.000 83.0 0.24 C09+27
05:19 0064 -1.237 0.384 -0.613 -16.803 6.032 0.000 78.4 0.21 C10+26
05:24 0065 -0.838 0.003 -0.249 -15.848 3.064 0.000 69.0 0.19 C10+13
05:28 0066 -0.310 -0.436 0.312 -17.321 0.937 0.000 54.2 0.17 C10-01
05:33 0067 0.152 -0.605 0.672 -20.518 -0.271 0.000 43.3 0.13 C10-12
05:38 0068 0.411 -0.585 0.902 -23.591 0.553 0.000 34.7 0.11 C09-21
05:49 0069 0.452 -0.171 1.083 -25.442 5.073 0.000 24.8 0.07 C09-30
05:53 0070 0.354 0.049 1.167 -27.354 8.995 0.000 19.7 0.06 C10-37
05:59 0071 0.410 -0.376 0.831 -22.038 1.168 0.000 33.5 0.05 C09-21
06:03 0072 0.361 -0.339 0.763 -20.515 0.665 0.000 35.8 0.06 C09-16
06:07 0073 0.141 -0.255 0.504 -16.654 -0.391 0.000 44.0 0.07 C09-09
06:11 0074 0.084 -0.226 0.487 -16.278 0.136 0.000 44.2 0.08 C10-12
06:18 0075 -0.398 0.040 0.074 -12.393 1.497 0.000 55.7 0.09 C10-01
06:25 0076 -0.700 0.163 -0.259 -12.478 2.683 0.000 63.9 0.09 C10+06
06:29 0077 -0.878 0.333 -0.474 -12.492 3.548 0.000 69.1 0.10 C10+13
06:34 0078 -1.446 0.914 -0.962 -12.858 8.151 0.000 80.6 0.11 C10+26
06:35   -1.395 0.917 -0.959 -12.717 7.619 0.000 80.5 0.11 192nm wfsc315
06:36   -1.335 0.890 -0.965 -12.912 6.976 0.000 80.4 0.11 415nm wfsc316
06:37   -1.338 0.884 -0.964 -12.882 7.028 0.000 80.2 0.11 138nm wfsc317
06:38   -1.319 0.886 -0.963 -12.755 6.831 0.000 80.1 0.11 161nm wfsc318
06:38   -1.269 0.866 -0.970 -12.874 6.306 0.000 80.0 0.12 331nm wfsc319
06:39   -1.227 0.864 -0.971 -12.790 5.863 0.000 79.8 0.12 268nm wfsc320

Notes: pointing correct twice mid-run. Last 7 data points are on the same source, taken through rising twilight. The last image at 06:39 I could only faintly see the star against the background. I am not sure if these data are useful, but I was wondering how much variability there is in the WFS loop. All collimation was set to go to the primary.

Done...closing.

Advanced Planning:

Execute TCS startup checklist:

Implicit Build16 TCS checkout tests (completed, all systems running):
Many of the features in Build 16 are implicitly tested by using the TCS SW.

  • TCS SW Release item 1, Verify the TCS services, subsystems, and GUIs launch properly. If the system comes up and is ok, this item has been tested.
  • TCS SW Release item 2, mcspu improved logging has been operational for the past week and is working ok since Nov 19th. This item has been tested.

Testing the TCS SW Build16

The new build comprises changes documented here: TCS SW Release Notes for Build 16

Explicit Build16 TCS checkout tests (Closed Dome tests):
Closed Dome tests:

  • Verify clocks on LUCIFER, AGW, TCS computers are in sync. Done 4:25MST. All computers checked out to be within 1ms of the GPS server on the mountain.
  • Issue a standard preset to check telescope functionality.
    OSA should do the following:
    • Verify LUCIFER is authorized on IIF GUI
    • cd /home/telescope/PresetCheck
    • ./preset.sh
    • Verify the preset completes as expected.
  • TCS SW Release item 4, check that LUCIFER can communicate with the new multi-client IRS. Both sides are using the previous version of IRS for Dec 03 observations.
  • TCS SW Release items 5 and 6, this is an item that affects LBC and should not affect Lucifer. If a preset succeeds for Lucifer this modification did not break Lucifer. An explicit LBC test can be conducted by stopping the OSS and issuing an LBC preset. If the LBC preset succeeds the modification is ok.
  • TCS SW Release item 7, PCS telemetry, the new three telemetry streams were tested and verified to work properly on the programmer's computer with a personal copy of the MySQL server. The PCS prints to the SYSLOG whether or not the three streams have successfully registered with the database. Regardless of whether or not MySQL is running, the PCS should continue to run successfully. Once the Build16 is operational on the mtn. the developer will verify the PCS telemetry is working on the mtn.

Check LUCIFER internal flexures
  • Take data for flexure compensation if we are running early enough. If not, take data prior to collimation run.

Beginning of the night pointing & collimation correction (10 minutes):
  • Use source list: Persson.cat
  • Select target at mid EL (50-70 degrees)
  • Preset in PARALLACTIC/TRACK mode
  • Adjust IE and CA; manually focus if necessary
  • Preset in POSITION/ACTIVE mode on axis
  • Allow system to collimate (WFS<400nm rms)

Explicit Build16 TCS checkout tests (Open Dome tests):

There are no explicit open dome tests. The two items that might be considered necessary to test on sky have already been tested off sky however on sky test would be of added value.

  • TCS SW Release item 3, GCS acquisition algorithm enhancement has been tested by running every acquired image to date through the enhanced algorithm. The enhanced algorithm, identified the star in every case.
  • TCS SW Release item 8, PCS radial correction scale factor. A radial correction scale factor has been incorporated for all coordinate interactions between the PCS and GCS; the scale factor generates "true" millimeters in the detector coordinate frame. Values which are used to compute the corrected/scaled values, as well as the uncorrected and corrected values themselves, are written to the SYSLOG for validation purposes. The values have been verified to be correct from a mathematical standpoint on both the programmer's machine and on the Tucson Test cluster; the programmer manually verified the computation using a calculator. The on-sky validation of this change is to acquire a guide star at a radially distant location such that a comparatively large scale correction must be applied.
  • TCS SW Release item 9, PCS affine transformation used for Lucifer. The programmer modified the affine transformation used for LUCIFER. Previously, the transform temporarily incorporated an "xflip" to accommodate the manner the LUCIFER software reads out the detector. The TCS programmer was directed to remove this temporary fix as it is understood by the LUCIFER team that they need to make this correction in their code. The manifestation of a mismatch will be when a request to move East is indicated, the telescope will move West.

SKIP THIS TEST FOR NOW!
MCS pre-processor algorithm verification (30 minutes):
  • Use source list: Persson.cat
  • Select target at mid EL (50-70 degrees)
    1. Preset ACTIVE mode on-axis to collimate
    2. Preset PARALLACTIC/TRACK mode to align DETXY to AZ/EL
    3. Start sequence with LUCIFER 120x1.0sec
    4. Offset 20 arcsec each in XY or NE
    5. Repeat steps 2-4 with larger offsets (60, 120, 240, 600 arcsec)
    6. Repeat step 1 as needed for collimation drifts
  • Note if dawdle-snap behavior remains
  • Data:
    LUCI# Observation Description
       
       
       
  • Results:

Focus LUCIFER/AGW on sky (30-45 minutes/field):
  • Can use in-focus images for astrometric solution?
  • Use source list: Persson.cat, LUCIastrom.cat?
  • Select target at mid EL (50-70 degrees)
  • In an xterm: ssh agwuser@agw-control
  • Preset in ACTIVE mode using off axis guide star
    1. Offset, take sky frame, return to preset position
    2. Offset AGW focus stage (0, -6, -3, 0, 3, 6, 0 mm from nominal).
      To set the AGw focus position, on agw-control window type:
      1. getfocus
      2. setfocus -p NN.nnn (absolute position in mm)
    3. Take min 30s exposure to average over seeing
    4. Repeat steps 2-3 stepping through focus sequence
    5. Derive best focus position (millimeters)
    6. Update focus in /lbt/tcs/current/etc/GCS/AIP_L.cfg
  • Repeat for at least 3 different stars at different ELs
  • Data:
    LUCI# Focus FWHM
         
         
         
  • Results:

Measure/Correct LUCIFER rotation angle (15-30 minutes/field):
  • Can use focus data for astrometric solution?
  • Use source list: LUCIastrom.cat
  • Preset to field, ACTIVE mode with off-axis guide stars
  • Take min 30sec exposure to average over seeing
  • Offset and take a sky exposure, same settings
  • Apply astrometric solution, determine rotator offset
  • Update LEFTZEROPOINT in the LUCIFER section of /home/telescope/TCS/Configuration/PCS/PCSInstrument.conf
  • Repeat on at least 3 fields, different elevations
  • Goal: Orientation better than +/-0.02 degrees
  • Data:
    LUCI# Field/Description PA(requested) PA(WCSfit)
           
           
           
  • Results:

Measure/Correct AGW-PCS Transformation (quick version, 15-30 minutes/field):
  • Prerequisites:
    LUCIFER accurately oriented on sky
    (PA=0 requested yields PA=0.00+/-0.02 achieved)
  • Use source list: Persson.cat
  • Use on-axis source with off-axis guide star that can be dithered 2+ arcmin in DET_Y
    1. Take min 30s integrations at each position
    2. Measure angular offset (0.02 deg = 0.35 pixels/1000)
    3. Update SFP_rotation in /lbt/tcs/current/etc/GCS/AIP_L.cfg
    4. Repeat steps 1-2 to confirm sign is correct
  • Repeat on 3 fields, different EL

Update Collimation Model (2+ hrs data collection, plus processing time):
  • Prerequisites:
    TELESCOPE IN THERMAL EQUILIBRIUM!
    DECENT SEEING (less than 1.25 arcsec FWHM on guider)
  • Confirm needed mirror positions, temps in LUCIFER headers
  • Use source list: collim.cat
  • Observe sources: 25(15) < EL < 87 if possible
    1. Preset in ACTIVE mode, on-axis
    2. Allow system to collimate: WFS<400nm rms
    3. Take 2s exposure with LUCIFER to record mirror information
    4. Repeat steps 1-3 as needed to cover EL-space
  • Observe minimum 1 full cycle through EL range, 12 sources,
    Prefer two cycles, 20+ sources.
  • Generate new collimation model (J.Hill?)
  • Implement new collimation model
  • Data:
    # LUCI# UT EL   # LUCI# UT EL
    01       * 02      
    03       * 04      
    05       * 06      
    07       * 08      
    09       * 10      
    11       * 12      
    13       * 14      
    15       * 16      
    17       * 18      
    19       * 20      
  • Results:

Update Pointing Model (2-4 hours data collection, plus processing time):
  • Prerequisites:
    TELESCOPE IN THERMAL EQUILIBRIUM!
    Using new collimation model from previous step
    Re-measure mechanical rotator center coords on LUCIFER
    • Take pair of TRACK mode images rotated 180 degrees apart
    • New rotator center: X Y
  • Update pointing and collimation prior to start, mid EL (50-70 degrees)
  • Use source list: Allstars10.cat, updated to include PA=0 in list
  • Turn on pointing log in PCS GUI
    Filename = ____________________
  • Use /home/LBTO/POINTING/Pointingstars5 to schedule
    1. In your working directory on an observer workstation:
      cp ~LBTO/POINTING/* . to copy program files
    2. Type ./Pointingstars5 at linux prompt to run program
    3. Enter the sequential reference field number [1-88]
    4. Preset to the source listed, TRACK mode, take 30s exposure.
    5. Repeat steps 2-4 for at least 24 points, 48+ would be better.
  • Stop pointing log in PCS GUI
  • Generate new pointing model (D.Thompson)
  • Implement:
    • Copy to ~telescope/TCS/Configuration/PCS/
    • Update LUCIFER entry for LEFTPOINTINGMODEL
      in PCSInstrument.conf
    • Re-authorize LUCIFER on IIF GUI to read in new pointing model
  • Test new pointing model
  • Data:
    # LUCI# Star ID   # LUCI# Star ID   # LUCI# Star ID
    01     * 02     * 03    
    04     * 05     * 06    
    07     * 08     * 09    
    10     * 11     * 12    
    13     * 14     * 15    
    16     * 17     * 18    
    19     * 20     * 21    
    22     * 23     * 24    
    25     * 26     * 27    
    28     * 29     * 30    
    31     * 32     * 33    
    34     * 35     * 36    
    37     * 38     * 39    
    40     * 41     * 42    
    43     * 44     * 45    
    46     * 47     * 48    
    49     * 50     * 51    
    52     * 53     * 54    
    55     * 56     * 57    
    58     * 59     * 60    
    61     * 62     * 63    
    64     * 65     * 66    
    67     * 68     * 69    
    70     * 71     * 72    
    73     * 74     * 75    
    76     * 77     * 78    
    79     * 80     * 81    
    82     * 83     * 84    
    85     * 86     * 87    
    88     * 89     * 90    
  • Results:

-- DavidThompson - 24 Nov 2008
Topic revision: r20 - 09 Oct 2009, NormCushing
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback