These are the reduction stages used in the analysis of 3-shuffle imaging (10'x5' subfields labelled A, B, C) of distant QSO fields in Nov 1998 (Boyle et al). A more generalized set of IRAF and Sextractor scripts is given here.

We assume that the TTF fields are already flat-fielded. First, the interference pattern from the sky background must be removed to allow faint detection and reliable photometry. Secondly, cosmic rays and ghosts must be exorcised effectively without reducing the signal-to-noise. Finally, the images must be aligned and cropped to remove bad edges due to shifting.

Step 1: Background removal. To remove the slowly varying background produced by interference rings from the night-sky, a novel-technique can be usefully applied to form a map of this background without knowing any other information. By shifting each TTF multi-frame image by a small amount (> diameter of sources on image e.g. 30 pixels) in 45degree directions to form a grid of 9 images each shifted relative to each other, the sources can be rejected. This is achieved effectively using IMCOMBINE in IRAF and the AVSIGCLIP rejection algorithm. At a 3sigma level of clipping and combining using the median (more robust than the mean), this provides an almost perfect background map. The main residual (besides that of very large saturated stars) is a "mottled" effect due from the shifting at a very low level. This is easily removed using a smoothing program such as BOXCAR at a similar level to the shifting. This smoothed background is then subtracted from each image in turn to leave "flat" image, aiding photometry.

Step 2: Cosmic rays & ghosts.For reliable results, cosmic rays and ghosts must be identified or better still removed altogether. The AVSIGCLIP algorithm was first tried at the 3-sigma level. This was found to be fairly reliable by most standards. However, some traces of cosmic rays could still be found which is only remedied by lowering the clipping level. The CRREJECT algorithm was found to be much more reliable as long as the correct readout noise (2 electrons) and CCD gain (1.11) are inserted into IMCOMBINE parameters. The ghosts were also removed with only a slight residual "ring" being noticed on the very brightest ghosts. However, the ghosts can easily be identified by comparing each narrow-band, or if shifted images are present, by looking for objects that have shifted twice the actual shift. Ghost are generally found in the upper frames (B+C) and not in the lower frame (A) - see figure 1.

Step 3: Aligning & cropping. Before photometry can be applied, the images must be checked for alignment. In the case of the BR0019-1522 field, the two sets of shifted images must be re-aligned before adding. The IMALIGN procedure in IRAF can be used to do this accurately. However, due to binning of pixels, fractional shifts can deteriorate the image, thus IMSHIFT was used to align to the nearest pixel. IMALIGN can still be used to calculate this automatically and thus give an estimate of the error. The error for this field was found to be less than 0.6 pixels, which is acceptable. Cropping the image is vital to remove effects of shifting. By cropping the images by exactly the correct amounts, the 3 frames can be made to align in pixel values. The charge shuffling using the MIT-LL 4096x2048 CCD (15m m pixels) causes a shift of 819 pixels between frames - the cropping values finally adopted are shown in figure 1. The 3 frames are labelled A,B,C from the bottom with increasing pixel numbers. Finally, the images can be added together to increase the signal-to-noise and provide subsets of images for later use in detecting spurious results/ghosts etc.
 

 
The flatfield step is probably the hardest to grapple with. How we tackle this really depends on the application, and whether it is the low order or high order modes which are causing the problems, and whether we are looking at point sources or extended sources.

a. Above,  the image was used to form its own superflat by dithering with respect to itself and then imcombining.

b.  Another approach for point source fields is to fit to everything, subtract, median smooth in order to form a low order mode flatfield.

c.  An ingenious method is to divide up the image into a rectangular grid and then, after median smoothing, choose the minimum value in each cell. This value for each cell is then used to generate a low order map using bicubic splines or whatever.

d.  High order structure, like fringeing, is usually removed with dome flats since the source is constant. Of course, this does not correct for OH fringeing in the red or at very high resolution. Take dome flats with the TTF for all the wavelengths (z values) you took during the night, and for same optical set up. That includes filter tilts, read speeds, etc. This should also take care of `extraneous etalon effects' sometimes seen in FP data. To date, we have not seen this in TTF data.

e.   If the fringeing is not too horrendous, dithering the images on the sky with lots of little exposures often works very well. You are almost always in the background limit with TTF.

f.  Scattered light: Flatfielding can be a real nightmare in the presence of scattered light from a bright star in the field, from moonlight or whatever. This is not such a problem for point sources since we can mask them out to get to the low-order background. But for extended sources, this can be a real problem. The best one can do is to form a model of the background and extrapolate over the region of interest. IRAF has functions like IMSURFIT and GEOMAP which can be useful. One should really be aware of the problem in observing time and move the target around the field (except that requires calibrations for the offset positions).
 

4.1 Cataloguing and photometry

 
To produce catalogues of objects and do the photometry, Sextractor v2.0 (Berlin & Arnouts, 1996) was found to be very useful due to features such as multi-deblending of objects, background-mapping, matching utilities and several other novel features. To be fair, the power of tunable filters in cosmology rests in part on good object finding codes like Sextractor.

There are several ways to search for objects with many different combinations of parameters. After some experimenting, it is found that the best way to detect all objects is using the co-added image (A+B+C). The optimum parameters are summarised in TABLE 2. A 1sigma level of detections is usually the best way to get > 99% of sources with very few spurious detections (<10%). This goes against what you would expect from counting statistics, but because Sextractor has several filtering mechanisms (e.g. gaussian filter, minimum detection area etc.), a 1sigma threshold produces very few spurious detections providing the background does not vary too quickly over the field.
 
 

Parameter	"Optimum values"	Comments
Detect_min_area	2-5	Does little to number of detections.
Detection_threshold	1	Critical value - 1sigma detects virtually all sources seen by eye unless the background is incorrectly sampled (see back_size)
Analysis_threshold	1	This is usually the same Detection_threshold.
Filter	Gaussian	Gaussian filter is the best for faint sources assuming a roughly Gaussian profile due to atmospheric seeing.
Deblend_Nthresh	32	Limited only by memory - low values should be avoided.
Deblend min_cont	0.001-0.01	Critical in determining the amount of deblending of objects. 0 means local variances will be treated as separate objects. 1 means no deblending.
Cleaning	Y(1) or N	Cleaning removes virtually all spurious detections very effectively. For forced photometry it should not be used, as the number of detections will vary with different frames due to different background calculations.
Mask type	Correct	Gives better magnitude estimates.
Phot_apertures	» 6	Depends on crowding and seeing.
Back_size	32 (64/128)	Important - the size of the background mapping. Low values means the background is poorly sampled. Large values means local variances cannot be established (i.e. bright sources).
Back_filter_size	3	3 is the best value. For very low back_size (e.g.8/16), a higher value can compensate in part to the poorly sampled background.
Assoc_radius	3	For matching objects, a 3 pixel allowance is ample.

TABLE 2: Parameters used in Sextractor v2.0









The catalogues produced can contain several parameters which can be later used for rejecting edge objects, spurious results and to compare differences between each band. The FLAGS parameter is produced by the various detection and measurement processes within Sextractor. The parameter increases from 0 depending on several factors such as is the object saturated, on the edge of the image, de-blended with another object etc (see Sextractor user guide). Other parameters include FWHM, eccentricity, background etc. which can be useful when objects have been initially identified.

Sextractor is able to compute several different types of magnitude calculation - isophotal, corrected isophotal, aperture, best and auto. Each has its advantages and disadvantages. However, for comparing objects in different fields, fixed aperture photometry is known to be the least biased as well as the level of noise to be estimated.
 

The main aim in this analysis is to observe differences in brightness of objects between the 3 frames. The narrow-bands are usually evenly spaced with at least one or more centred on an emission lines Doppler-shifted to that wavelength. Of course, the brightness differences will depend on how broad the line is and how accurate the red-shift actually is.

Blinking the frames using XIMTOOL can often reveal some of these differences by eye. The usual way of displaying magnitude differences is to produce a plot of magnitude difference against some reference magnitude. The reference magnitude should not be biased thus it is usual to use an average magnitude involving the magnitude differences. For example, if magC- magB is on the y-axis, the x-axis should be (magC+magB)/2 to keep the plots symmetrical. For the majority of the plots the co-added magnitude (A+B+C) was taken.

Fixed aperture photometry is generally preferred for this analysis as it is less biased than isophotal magnitudes. Smaller apertures give more sensitivity to faint objects but increase the errors. Large apertures can give inaccurate results in crowded fields. A 6 pixel aperture was used, corresponding to about twice the seeing disk.

There are several errors which occur in the estimate of the magnitudes which are vital to knowing how real any difference might be. Usually the signal-to-noise ratio is given as:



where N is the number of counts from the object/sky and RN is the readout-noise of the CCD. This is the true SNR due to Poisson noise (a 1/ÖN). However, there are extra errors which often dominate the Poisson noise. There is an error in the estimation of any offset between frames which may vary across each field i.e. may not be constant. Secondly, the process of deblending which allows Sextractor to work so well can add an extra error. It is therefore wise to assume that the errors given by Sextractor are accurate.

Offsets need to be applied to the magnitudes calculated due to variations between the 3 panels. It is assumed that this is a constant due to previous flat-fielding. This is done by forming a catalogue of bright but unsaturated objects from each of the frames and correcting the averages to one of the frames.

Upper limits to detections can be added by using the matching facilities in Sextractor to find objects detected only in one particular frame, and would be useful indicators of the detection limit. The detection limit is found by finding the turnover in a histogram of number vs. magnitude (see e.g. figure 11).