next up previous contents
Next: Spectral extraction: dofibers Up: FLAIR IRAF Guide Previous: Data Transfer

Basic CCD Reduction

This section describes the basic steps in preparing the CCD images for spectal extraction. The IRAF packages imred and ccdred should be loaded first.

Combining the bias frames: zerocombine

The next stage of the process is to prepare the bias frames for use. These are required in order to remove any structure in the electronic pedestal (bias) of the CCD. In anticipation of this need, you will have taken many (at least 25 so they are not dominated by readout noise) ``zero second'' exposures which need to be combined into a single image, hence the current task zerocombine.

In the example below, listb is a text file containing the names of the bias frames to be combined with one name per line of the file. If there are very few such frames, you can give zerocombine the image names directly via eparam. The output is a single image called Zero. Most parameters can safely be left at default values; however ccdtype needs to be set to `` '', since in general, the image type is not specified correctly in the image header. The image type needs to be correct for the next stage, so correct it if necessary (use hedit to set the ``imagetyp'' of Zero to ``zero'').

lparam zerocombine
        input = @listb          List of zero level images to combine
      (output = "Zero")         Output zero level name
     (combine = "average")      Type of combine operation
      (reject = "minmax")       Type of rejection
     (ccdtype = "")             CCD image type to combine
     (process = no)             Process images before combining?
      (delete = no)             Delete input images after combining?
     (clobber = no)             Clobber existing output image?
       (scale = "none")         Image scaling
     (statsec = "")             Image section for computing statistics
        (nlow = 0)              minmax: Number of low pixels to reject
       (nhigh = 1)              minmax: Number of high pixels to reject
       (mclip = yes)            Use median in sigma clipping algorithms?
      (lsigma = 3.)             Lower sigma clipping factor
      (hsigma = 3.)             Upper sigma clipping factor
     (rdnoise = "0.")           ccdclip: CCD readout noise (electrons)
        (gain = "1.")           ccdclip: CCD gain (electrons/DN)
       (pclip = -0.5)           pclip: Percentile clipping parameter
        (mode = "ql")

Combining the flat field frames: flatcombine

A second type of calibration frame is the flat field frame. A flat field is an image of a uniform intensity distribution, e.g. the dome flat-field screen or the twilight sky, (the latter is to be preferred). This is used to determine and correct for the relative efficiency of the different fibre apertures. It is also used to identify the fibre apertures. Flatcombine combines any number of flat frames in similar fashion to zerocombine, the ouput file in this example being called Flat. There is one important difference though. Since the exposure time is non-zero, there will inevitably be cosmic ray events or tracks (CREs) on the image. Flatcombine uses an algorithm to weed these out. The recommended choice for the reject parameter is ``avsigclip''. This uses an average sigma clipping algorithm to reject pixels whose values deviate significantly from the mean, and eliminates CREs with minimal loss of data.

As with zerocombine, ccdtype should be set to `` ''. Note also that processing the flat frames before combining is unnecessary, since this is done by the task ccdproc (see below).

lparam flatcombine
        input = @flist          List of flat field images to combine
      (output = "Flat")         Output flat field root name
     (combine = "average")      Type of combine operation
      (reject = "avsigclip")    Type of rejection
     (ccdtype = " ")            CCD image type to combine
     (process = no)             Process images before combining?
     (subsets = yes)            Combine images by subset parameter?
      (delete = no)             Delete input images after combining?
     (clobber = no)             Clobber existing output image?
       (scale = "mode")         Image scaling
     (statsec = "")             Image section for computing statistics
        (nlow = 1)              minmax: Number of low pixels to reject
       (nhigh = 1)              minmax: Number of high pixels to reject
       (mclip = yes)            Use median in sigma clipping algorithms?
      (lsigma = 4.)             Lower sigma clipping factor
      (hsigma = 4.)             Upper sigma clipping factor
     (rdnoise = "0.")           ccdclip: CCD readout noise (electrons)
        (gain = "1.")           ccdclip: CCD gain (electrons/DN)
       (pclip = -0.5)           pclip: Percentile clipping parameter
        (mode = "ql")

Now you can save disk space by discarding the redundant bias and flat frames ( imdel @blist; imdel @flist).

Processing the images: ccdproc

We now come to a very important stage in our reduction of the FLAIR spectra: ccdproc. This task incorporates a veritable host of minor functions, all combined into the one easy-to-use task. After passing through ccdproc, your data will be ready for spectral extraction.

The task removes the electronic zero level first by subtracting the output of zerocombine from each frame (to remove any constant structure) and secondly by subtracting the average over the columns in the overscan region (to remove any frame-to-frame variations in the average zero level). A fit (generally a constant) is performed on the overscan region of each image as a function of line number and is then subtracted from all columns of the data part of the image. Note that when FLAIR images are displayed in IRAF, the short (slit) dimension is horizontal and the long (dispersion) dimension is vertical. The x-coordinate (column no.) then measures horizontal (slit) distance and the y-coordinate (line no.) gives vertical (dispersion) distance. After subtraction of the overscan region the image is trimmed to leave just the part containing useful data.

It is necessary to specify the portions of the image containing data and the overscan with the parameters biassec and trimsec; these may be determined by inspection with a task such as implot. The format for biassec and trimsec is:

            [start column:end column,start line:end line]

Some example values for biassec and trimsec are included in the parameter listing below. These serve as an approximation, but you should check (with implot) to be sure.

The ccdproc task can also correct bad pixels by interpolation (if fixpix=yes). A file listing known bad pixels in the first CCD (before 1995 June 1) is provided with the package in flair$badpix.dat. The new CCD has very few cosmetic flaws and no file is provided for it.

There are quite a few other things ccdproc can do, for example making dark-current corrections. However, this is usually not necessary with the CCD used with FLAIR. It is also possible to make a true flat-field correction, but this is not done because it is difficult to illuminate the CCD uniformly and this is not usually necessary for spectroscopic observations.

Hint: to display the full images try the image display configuration imt2 (800x800). To define this type ``reset stdimage = imt2'' in IRAF before displaying an image with ``display image 1''.

lpar ccdproc
       images = @list1          List of CCD images to correct
     (ccdtype = "")             CCD image type to correct
   (max_cache = 0)              Maximum image caching memory (in Mbytes)
      (noproc = no)             List processing steps only?
      (fixpix = yes)            Fix bad CCD lines and columns?
    (overscan = yes             Apply overscan strip correction?
        (trim = yes)            Trim the image?
     (zerocor = yes)            Apply zero level correction?
     (darkcor = no)             Apply dark count correction?
     (flatcor = no)             Apply flat field correction?
    (illumcor = no)             Apply illumination correction?
   (fringecor = no)             Apply fringe correction?
     (readcor = no)             Convert zero level image to readout correction?
     (scancor = no)             Convert flat field image to scan correction?
    (readaxis = "line")         Read out axis (column|line)
     (fixfile = "flair$badpix.dat")   File describing the bad lines and columns
     (biassec = "[390:397,3:577]") Overscan strip image section
     (trimsec = "[12:360,3:577]") Trim data section
        (zero = "Zero")        Zero level calibration image
        (dark = "")             Dark count calibration image
        (flat = "")             Flat field images
       (illum = "")             Illumination correction images
      (fringe = "")             Fringe correction images
  (minreplace = 1.)             Minimum flat field value
    (scantype = "shortscan")    Scan type (shortscan|longscan)
       (nscan = 1)              Number of short scan lines
 (interactive = no)             Fit overscan interactively?
    (function = "legendre")     Fitting function
       (order = 1)              Number of polynomial terms or spline pieces
      (sample = "*")            Sample points to fit
    (naverage = 1)              Number of sample points to combine
    (niterate = 1)              Number of rejection iterations
  (low_reject = 3.)             Low sigma rejection factor
 (high_reject = 3.)             High sigma rejection factor
        (grow = 0.)             Rejection growing radius
        (mode = "ql")

The arc frames

Calibration arc frames should be taken before and after each set of observations to provide the necessary wavelength calibration for your data. It is very important to check these for any shift during the night (use implot to draw cuts through the images). The FLAIR spectrograph is very stable so any large shifts ( 1 pixel) are unlikely. Small shifts are corrected by the IRAF spectral reduction task by interpolating between the arcs taken before and after the observations.

If you have a number of ``identical'' arc images taken together these may be combined using the task combine as described in the next section in order to remove any CREs:

        combine @arc_list arc_neon combine=average reject=avsigclip

There are several different arc lamps available, the most common being mercury-cadmium (Hg-Cd) and neon (Ne). The choice of lamp(s) used depends on the spectral range observed and resolution. Note that in some cases (e.g. low-resolution observations of galaxies with a dispersion of 6.1 Å per pixel) both the Hg-Cd and Ne lamps are used and the two resulting images may be combined using the task imarith:

        imarith arc_neon + arc_hgcd arc1

Combining the object frames: combine

The data frames may now be combined: this is done to increase the signal-to-noise of the spectra as well as to remove CREs using a suitable clipping algorithm. It is wise to check all the frames first by displaying them (e.g. on SAOimage). There is a limitation to this approach in the case of fibre spectrographic data: during the night there are often large variations between frames as the relative intensities of the spectra change. These are caused by variations in seeing, airmass or field alignment and may result in real data being removed by the clipping algorithm.

A suggested method is to combine the images in consecutive groups of two or three chosen so that the images in a given group are quite similar. The ``crreject'' algorithm is suggested: this removes the CREs effectively when combining as few as two images by using the properties of the CCD (gain=1, read noise=10). To avoid removing real data in the clipping stage, large thresholds are suggested (e.g. lsigma=7, hsigma=7). As a further precaution it is a good idea to combine the data a second time without sigma-clipping, then take the difference (with imarith) of the two combined images. The difference will show you what was rejected: it should look random; if any spectra are visible then the thresholds were set too low.

Each combined image may then be processed separately by the spectral extraction package and the output spectra finally all combined with the task scombine. This does not involve too much extra work because the interactive aperture editing and wavelength calibration steps are not repeated. When combining images it is a good idea to put the middle one of the chronological sequence first in the list, giving the output image a timestamp near the middle of the effective exposure. This gives a better result if the wavelength calibration is interpolated (as is the default below) from arcs at the start and end of the night.

lpar combine
        input = "@list2"        List of images to combine
       output = "fornax2"       List of output images
      (plfile = "")             List of output pixel list files (optional)
       (sigma = "")             List of sigma images (optional)
     (ccdtype = "")             CCD image type to combine (optional)
     (subsets = no)             Combine images by subset parameter?
      (delete = no)             Delete input images after combining?
     (clobber = no)             Clobber existing output image?
     (combine = "average")      Type of combine operation
      (reject = "crreject")     Type of rejection
     (project = no)             Project highest dimension of input images?
     (outtype = "real")         Output image pixel datatype
     (offsets = "none")         Input image offsets
    (masktype = "none")         Mask type
   (maskvalue = 0.)             Mask value
       (blank = 0.)             Value if there are no pixels
       (scale = "exposure")     Image scaling
        (zero = "none")         Image zero point offset
      (weight = "exposure")     Image weights
     (statsec = "")             Image section for computing statistics
  (lthreshold = INDEF)          Lower threshold
  (hthreshold = INDEF)          Upper threshold
        (nlow = 1)              minmax: Number of low pixels to reject
       (nhigh = 1)              minmax: Number of high pixels to reject
       (mclip = yes)            Use median in sigma clipping algorithms?
      (lsigma = 7.)             Lower sigma clipping factor
      (hsigma = 7.)             Upper sigma clipping factor
     (rdnoise = "10.")          ccdclip: CCD readout noise (electrons)
        (gain = "1.")           ccdclip: CCD gain (electrons/DN)
    (sigscale = 0.1)            Tolerance for sigma clipping scaling correction
       (pclip = -0.5)           pclip: Percentile clipping parameter
        (grow = 0)              Radius (pixels) for 1D neighbor rejection
        (mode = "ql")

Aperture identification: reformat

Before proceeding, it is necessary to create an aperture identification file specifying which apertures are sky and which contain the objects. This is a file consisting of three columns of data, e.g.

1   0   999   
2   1   43   
3   1   104   
4   0   999   
5   1   20   
6   1   7   
7   1   67   
8  -1   96   
9   1   8   
10  0   999

The columns are, from left to right, fibre number, object type, and object identification number. The object type specifies what the fibre is looking at: 1 for an object, 0 for sky. If the fibre is broken or ``blanked off'', then the object type is -1; any such fibre is ignored. The object identification number is a unique number assigned by you to each object. If it's a sky fibre then a value of 999 is usually employed. Plateholder 14/5 has 92 100m fibres and plateholder 14/6 has either 73 m fibres or 70 m fibres.

The aperture identification file can be typed in from the fibring-up records. Alternatively, it can be created directly from the log file on the AUTOFRED PC used to fibre-up. The file AF.LOG can be transferred to another computer by floppy-disk (ask the Schmidt staff for assistance). The local IRAF task reformat may then be used to reformat the log file automatically. Note that reformat does not identify sky fibres correctly unless they are labelled ``999''. It also correctly identifies broken fibres labelled ``-1''.

Trimming the flat field frame: triangle

One use of the flat field image is to calibrate the (relative) efficiencies of the different fibre apertures. The differences in efficiency are caused by 'dirty' prisms, ferrule/prism misalignment, and poor seating of the ferrules (all of which can vary with each fibring up of a field), inherent variations in the transmission of each fibre (constant) and telescope vignetting (a function of position in the field).

These variations are corrected by taking a flat field in which every aperture is exposed in principle to the same amount of light. The resulting signal in each fibre aperture is then summed (in the dispersion direction) to measure its efficiency. Unfortunately, the ``slit'' may not be perpendicular to the dispersion axis, thus there may be a tilt in the images. This is automatically removed during the wavelength calibration. However, throughput correction takes place before this. This means that the fibres on one side of the image would have extra signal. To correct this (to first order), triangle replaces the pixel values in a triangle at the top of the image with zero. The main parameter is ``steps'' which measures how far a line of constant wavelength (e.g. a sky feature) move from one side of the image to the other due to the tilt. This can be up to 6 pixels, although you should check this for each observing setup. For features that slope down going left-to-right ``steps'' is positive; for features that rise it is negative. The throughput image name is created from a flat field frame, such as Flat, and is conventionally named with the extension t (in the example below, Flatt).

lpar triangle
        input = "Flat.imh"      Input mages
     (outputx = "t")            Output extension
      (newval = 0.)             Replacement value
       (steps = 6)              Number of steps across image
        (show = no)             Print operations?
     (imglist = "tmp$flairf445a") 
        (mode = "ql")



next up previous contents
Next: Spectral extraction: dofibers Up: FLAIR IRAF Guide Previous: Data Transfer

Mon Aug 19 16:47:38 EST 1996