otatool

An "OTA" is an Orthogonal Transfer Array CCD composed of 8x8 cells of (typ.) 500x500 or 600x600 each. We have developed a tool to generate some quick image statistics and create a fix color stretch quicklook JPEG from an OTA image saved as a standard Iraf Multi-Extension FITS (MEF) with 64 image extensions. With the proper keywords, otatool may be useful on other MEF images as well.

% otatool -h

Currently produces the following help summary:

OTA FITS Manipulation Tool v1.23 (svn 3567, Mar 13 2009)
usage: otatool [options] < inputfile.fits > outputfile.fits

where [options] can be (default values shown in [...])
        open=   Specify an open-filter or default input FITS file
        blue=   For making color images, specify a blue-filter FITS file
         red=   For making color images, specify a red-filter FITS file
        dark=   Specify a dark calibration FITS file to subtract
       j*peg=   [false] Output full sized JPEGs by name of each extension
     jmosaic=   [false] Generate a mosaic JPEG of all cells/extensions
      jthumb=   [false] Generate thumbnail JPEGs (1/8 size) of everything
      jnoise=   [false] Generate RMS noise diag JPEG mosaic of full OTA
      jc*map=   [gamma] Colormap (gamma, rainbow, or terra)
      jgamma=   (Obsolete, ignored.)
   jq*uality=   [95] JPEG quality factor (1 to 100)
    ext*name=   Specify a single extension to extract, by its EXTNAME
          x0=   Corner pixel x-coord of a cropping box.
          y0=   Corner pixel y-coord of a cropping box.
          xs=   Size in x of a region to crop.
          ys=   Size in y of a region to crop.
     prescan=   Override any row-prescan value in the FITS files
   bias*norm=   [true] Normalize all overscan bias levels at 250 ADU
  fancy*bias=   [false] Custom bias filter (not valid for science!)
     ped*sub=   [true] Subtract pedestal if present (true/false)
    gaussian=   [0.0] If >0, apply a blur of that FWHM (before scaling)
  cut*levels=   [auto,auto] Force low and high ADU cut levels
   ramp*jpeg=   [off] Just output a color-bar intensity ramp
        flip=   [false] Flip image top-to-bottom
  comp*osite=   [false] Convert from a MEF to a basic, composite FITS
      median=   [false] Display median values of all cells
       stats=   [false] Generate pedestal, bias, sky level and noise stats
ponstats*file=   Text input file with XRAY (gain) and LED (linearity) stats
up*date_keywords=   [false] Attempt to update FITS keywords with stats
   statsfile=   Text file to create which contains statistics
   cplotfile=   Create color plot from existing stats file
    cplotccd=   Name of the ccd, such as "ota34"
    cplotvar=   [rmsov] Variable to color-plot
(Do not type the *.  Letters after the * are optional.)

Dark Subtraction

Some day, otatool is supposed to be able to do a scaled DARK subtraction from the image before generating JPEGs so they look nicer, but this is untested and probably not working correctly.

option	description
dark=darkmef.fits

True Color JPEG Generation

The otatool utility can generate a realistic JPEG image of a scene if provided with FITS files with neutral filter, red, and blue (without anything in the scene changing.) This functionality has not been used or tested recently and probably does not work correctly. This just documents the options that belong with this mode of operation.

option	description
open=mef0.fits
blue=mef1.fits
red=mef2.fits

False Color JPEG Generation

More commonly, we use otatool to generate a false color image of a single FITS frame. That mode of operation produces JPEG files with fixed names, so we run it in a separate subdirectory and then rename those output files. (In the future, we may add another command line option to control the output names.) Typical invocation for this mode of operation is:

% otatool jpeg=true < mef.fits
% ls
< ... produces *.jpeg where "*" is the EXTNAME found in the FITS file. >

option	description
jpeg=true	Cause full 1:1 JPEGS to be generated for each image extension
jmosaic=true
jthumb=true
jcmap=	Choose between colormaps called gamma (the default) rainbow, and terra. The gamma colormap is special because it modulates the brightness (using the rainbow colormap as a start) so that the steepest changes in brightness occur at the median of the image (see cutlevels=, below.)

Statistics Generation

You can use otatool to calculate the RMS of the overscan, or a part of the image in each extension and report the results to a "stats" text file. This can happen at the same time as JPEG generation, or instead. (Just specify all the relevant options and otatool will carry out the operations requested.) Options related to statistics calculation are:

option	description
stats=true	Generate statistics
statsfile=foo.txt	Where to write the calculated numbers
prescan=	Normally, our practice is to remove prescan with the controller (using clv prescan=3 or 4 for OTAs, or prescan=50 for e2v for example.) If the image still contains prescan pixels, you must tell otatool how many there are with this option so it can skip them.

Here is a description of the statistics values calculated in the stats files:

stats var	description
ped=	If a readout was performed with "bpp=32" and the FITS file was saved with "bpp=32" then otatool can report on the mean pedestal level in the pedestal image. Usually, the pedestal image is subtracted from the data before saving but this can be a good diagnostic. Pedestal levels should be within a few 100 ADU of 5000 (or whatever value you use for "readcal".)
bias=	Measures the bias level, in ADU. This assumes otatool is looking in the right place for overscan. See "prescan=" above, and notes below.
back=	Measures the mean level (above bias) in a portion of the image area. The portion used is currently hardcoded into otatool.c
rmsped=	If you have a 32bpp STARGRASP FITS file (see ped= above) then this statistic reports the RMS in ADU of the pedestal. Expect this to be quite high in many applications. This is the uncorrelated sampling noise.
rmsvid=	If you have a 32bpp STARGRASP FITS file (see ped= above) then this statistic reports the RMS in ADU of the uncorrelated video sample. Like rmsped=, it could also be quite high (100s of ADU sometimes) but should be similar to the rmsped value.
rmsov=	This is the RMS of the subtraction of (vid-ped) which is the 16-bit value normally delivered as the image, for pixels in the serial overscan region. See notes below.

Additional RMS values for image regions (intended to be used on BIAS readouts only) are available if otatool has been compiled with "-DDO_STREAKY" in the Makefile. These stats are:

stats var	description
rmsmask=	The RMS in ADU of part of the image (region used is hardcoded in otatool.c) after masking out any bright chip defects.
rmsrow=	The RMS in ADU of the same region, after applying a "row-by-row" bias correction to each pixel row individually.
rmsplane=	The RMS in ADU after the above treatment by rmsrow, but after fitting a plane to the image to take out any overall horizontal slope.
rmstilt=	The RMS in ADU after fitting a sloped bias correction to each individual row of pixels.

NOTE: The region which otatool uses for "overscan" statistics and the window where the rmsmask/row/plane/tilt are taken are fragile, and hardcoded into the program! Most commonly, they can be confused by incorrect "prescan" setting. otatool is usually only used on data that contains no serial prescan pixels! The STARGRASP controller will read them and not transmit them if the clvset clocking pattern is set up with prescan=N.