Configure Cookbook

Cookbook for 2dF Target Preparation using Configure

Observations files for 2dF are prepared using the Configure software.  This CookBook is aimed at providing a basic introduction to the configuration process. An updated manual will be available shortly. In the meantime, please contact your support astronomer with any questions about the software or problems with configuration.

Contents:

Creating a Configure Input File

You will first need a text file containing 500-800 proposed targets.  If sparse pre-sampling of your sample is required to reduce it down to 500-800 targets, options for this are described in the Guide to Complex Configurations below.

Configure reads in ascii .fld files. An example .fld file is available to illustrate the basic format example.fld. In constructing your own .fld files the following considerations need to be made:

    • Science targets: No more than 800 targets and these should cover a relatively small range in target magnitude (less than 3 mags is the standard constraint, but talk to your support astronomer if you require more detail here).

    • Calibration sources: If required, these should be set to Priority 9 in the .fld file with the priority of all science targets shuffled to lower levels so that the calibrators are always allocated.

    • Sky fibre positions: You will need 20-30 sky fibres in the observation, so 50-100 possible sky positions should be enough. Eyeball the sky fibre positions to check they are actually blank regions.

    • Standard star calibrators: We have had some success recently in including a small number (1-2 objects per configuration) of standard star calibrators in 2dF/AAOmega fields. These must be chosen to be faint, to avoid contaminating science spectra. Drawing the calibrators from the recent sample of White Dwarfs and Hot Sub-Dwarfs of Eisenstein et al. ApJS, 2006, 167, 40 from SDSS has worked well. Absolute flux calibration is not possible with a fibre system such as 2dF/AAOmega, due to the unquantifiable aperture losses in any given observation, but including a standard star in each field plate observation can improve the quality of internal spectral calibration, and monitor data quality during a run. All caveats relating to astrometric accuracy apply to calibrator data as well as science and guide data.
    • Guide stars (Fiducials): These are crucial to the success of your observations so pay careful attention here.

      • The 2dF positioner has 8 guide bundles available. You should aim to use all 8. This will require 20-30 candidate guide stars distributed across the field plate to prevent guide star selection compromising science fibre placement.

      • In dark sky conditions, guide stars should be 14-14.5mag (B,V or R mags on the Vega or AB system). Up to 12th magnitude stars may be needed during bright-of-moon, (and check the location of the moon relative to your fields).

      • Guide stars MUST be on the same astrometric system as your targets.

      • Guide star spatial distribution must match that of the stars.

      • The range of guide star magnitudes should be small, preferably less than a 0.5 mag spread. A greater spread will cause problems due to the dynamic range of the camera used to guide with.

        GUIDE STAR WARNINGS:

      • Simply selecting some bright guide stars from SIMBAD or GSC is NOT going to work, your astrometric solution MUST be the same for the guide stars AND the targets, and good to 0.3arcsec or better. This is a requirement for 2dF positioner observations.

      • UCAC-4 and 2MASS sources have proved successful in recent years, although the USNO survey seems to be somewhat inconsistent (probably due to plate boundary effects).

      • SDSS is an obvious source of guide stars. Marginally saturated stars, which do not suffer obvious defects on examination, have been found to still give excellent results with 2dF/AAOmega (the SDSS astrometic data for these objects actually comes from smaller edge CCDs so the stars do not actually saturate in their astrometric reference frame).

      • Eyeball your guide stars. Reject galaxies, reject binaries, reject objects with junk magnitudes. 

    • Assigning Specific Wavelengths to specific targets: The telescope's Positioner GUI also handles atmospheric refraction effects when working out the positions of fibres on the field plate - including the effects caused by different observation wavelengths. Normally a single wavelength is chosen for all fibres and is applied by the support astronomer. However, it is possible that you may prefer to have fibres configured for different wavelengths. It is now possible to specify up to 9 different wavelengths in the .fld file (also shown in the example.fld file):

      • In the field header of the .fld file, specify a set of wavelengths of the form:

        WLENn wavelength

        where n is an integer in the range 1-9 and wavelength is a wavelength specification in Angstroms.

      • For each program object, P, you may now append a wavelength selector of the form P_wn where n is in the range 1 to 9.

      • Once this is configured the wavelength specified in the 2dF positioner GUI by the support astronomer becomes the default wavelength (used when a specific wavelength is not specified and for Sky objects). All fibres with specific wavelengths will be positioned optimally for that wavelength and the observed .fits files have a "WLEN" field in the binary table associated with each object to indicate the actual wavelength the object was configured for.

    • A warning on the use of target priorities: The SA algorithm is very good at allocating targets based on the 9 possible priority levels (9 is highest priority). However, the user should exercise some restraint when using the available levels. Using all of the available priorities to derive a complex priority selection function will almost always yield very limited returns at the expense of usability.  For most programs the number of targets in a given .fld file must be restricted (as described above) in order to allow the configuration process to be completed in an appropriate amount of time (~20mins). A field that is stacked with a large number of low priority targets will take a long time to configure. If these targets are indeed low priority then the user should consider carefully whether their inclusion in the .fld file is worth the overhead in configuration time they will incur.

Running Configure

Once launched, Configure asks you to select the instrument you wish to configure for (Figure 1; 2dF-HERMES, 2dF-AAOmega, 2dF, 2dF-old-404, FLAMES, 6dF).
 
Configure Instrument Selection
Figure 1: Instrument selection window
 
Once the instrument has been selected several windows are opened.  The "Basic Sequence" window (Figure 2) is the starting point.
 
Configure Basic Sequence
Figure 2: Basic Sequence window
 
From the basic sequence you:
    • Select the field plate to prepare the configuration for (plate 0, plate 1 or plate 2 which can be observed with either plate 0 or plate 1).
    • Apply a magnitude filter (this is very rarely used).
    • Open the .fld  file to be configured.
    • Select the fibre combination to be configured (this is rarely changed from the default "All Fibres").

Once these options are set and the .fld file opened, the "Allocate" button can be clicked.  This opens the "Allocation" window (illustrated in Figure 3) from which configuring parameters can be set.  The default settings are fine for the majority of programs but more detail on the available parameters, including hidden Expert options, is given below.

 
Once the chosen parameters are selected, the OK button can be selected.  This initiates the configuration which can be followed in the main Configure window. 
 
When the Configuration is complete the simulated 2dF window will illustrate the configured fibre positions. 
 
At this stage it is a good idea to check the numbers and distributions of guide stars configured and also that the configuration is observable over a range of hour angles.  This can be checked using Commands menu item "Check over Hour Angle" and checking over 4 hours. Those fibres that are flagged as having conflicts over this time should be reallocated or deallocated by clicking the relevant fibre in the simulated 2dF window and using the Commands menu to deallocate and/or reallocate the fibre. 
 
Once the configuration is complete the binary file for input into the telescope should be saved, using "save as SDS file" from either the "Basic Sequence" window or the File menu. You are now ready to observe these targets.

Options for Allocation

    1. Annealing. This governs how quickly the annealing routine cools during the allocation process. The Standard setting is generally fine.

    2. Weight close pairs: ThetaMin; ThetaMax. In some circumstances one may wish to give additional weight to closely packed targets, at the expense of overall target yield. These allow this to be setup, but beware of the odd effects it will have on your allocation. This option has not been extensively tested to date.

    3. Cross beam switching. If the observation requires Cross Beam Switching (CBS) between pairs of fibres, then the user should first generate the paired target positions using the menu option Commands->Generate CBS pairs and then set the CrossBeamSwitching flag. This gives additional weight to targets which are successfully allocated pairs of fibres, at the expense of overall target yield.

    4. Straighten fibres. This gives increased weight to allocations which have fewer fibre crossovers. While this will have some impact of target yield, the effect is small/undetectable for most source distributions and results in fields that typically require fewer fibre parks between configurations, hence reconfiguration is faster (perhaps by 10minutes, ~20%, in some cases). Fig. 13 of Miszalski, Shortridge and Saunders et al. (MNRAS, 2006, 371, 1537), (arXiv:astro-ph/0607125) shows the effects of this straightening. It can have adverse effects on target priorities and so the concerned user will need to experiment with this option to determine the optimal solution.

    5. Collision Matrix. It is occasionally useful to save the matrix of fibre collisions which has been calculated for this field. This enables quick restarts of the software later on. This file can however be rather large.

    6. Enforce sky quota. This option forces the allocation of the requested number of sky fibres. This can result in subtly lower target yields for some fields, although the effect is small/undetectable for most source distributions (accepting that the full sky quota is allocated to skies). Most datasets will be of little value with less than 15 sky fibres. 20-30 fibres is more typical for most projects.

    7. Peripheral weighting for Fiducials. This gives enhanced weight to selection of stars towards the edge of the field, which is typically beneficial for acquisition, and prevents all of the fiducial stars being crowded into a small area of the plate, as can happen with the SAconfigure algorithm.

    8. Weight fiducial target pairs. For CBS observations one may wish to allocate the fiducial fibres in pairs in order to guide in both positions of the beam switch. Setting this flag gives extra weight to paired fiducial allocation. Note: it is often more efficient in terms of fibre allocation for the user to allocate fiducials by hand but to ensure that half of the fibres (e.g. 50, 150, 250 and 350) go to position A guide stars, while the other half (e.g. 100, 200, 300 and 400) go to position B guide stars. There is no requirement that these stars be the same set in the A and B positions.

    9. Number of background threads to use. The calculation of the fibre collision matrix is very CPU intensive. On a modern multi CPU machine Configure can hijack all of the available CPUs and run a number of background threads, this vastly reduces the allocation time. For a single CPU machine, there is nothing to gain here.

    10. On-the-fly collision calculation. By default, the the collision matrix is calculated in full in advance of the annealing (this is the way Configure-v7.4 operated when SAconfigure was first introduced). An alternative is to calculate it on-the-fly. This ensures that A configuration is achieved as quickly as possible. This configuration will be HIGHLY sub-optimal. The longer the process is allowed to run, the greater the region of parameter space that is investigate and the the better configuration will be. In the limit of the annealing process, the two approaches will produce identically good configurations, and will take identically long to reach this point. There is therefore often little point in doing the calculations on-the-fly. In fact this option may allow inexperienced/inpatient users to produce sub-optimal configurations. It can however, be used in cases where a pretty good configuration is needed rapidly. Note: the original Oxford configuration algorithm, which can be used instead of the annealing by running the configureTrad command, will be far quicker.

    11. Number of Sky fibres. The indicated number of sky fibres will be assigned (but see the note above on enforcing the sky fibre quota).

    Allocation menu
    Figure 3: Allocation options in Configure

     

    Additional Expert allocation options

    These options can only be accessed via the Expert user mode which one activates via the toggle setting in the Options menu. These settings are generally for support astronomers and expert users. These are illustrated in Figure 4.

    1. Fibre clearance, Button clearance and pivot angle. These options are mainly for the 2dF support staff. If you do not know what they are used for then you should not adjust them. Note that the 2dF robot has safe values HARD WIRED into the system and so a configuration which is outside these bounds will be flagged as INVALID at configuration time. These settings should only be used to restrict the values to tighter constraints for reasons that are beyond the scope of this web page.

    2. Random Seed and Percentage of allocations sampled. If one needs to configure more quickly, e.g. if the field is pathologically complex (usually centrally condensed or with heavily clustered targets) and one cannot alter fibre allocations as described above (and on the complex configurations page), then it is possible to sparse sample the collision matrix and speed up the process. The details of this option are beyond the scope of this web page and should be discussed with your support astronomer. The principle is, for such configurations, that the slow speed is caused by the large number of rather similar configuration that are available (in essence many objects could be configured with many different fibres without changing the basic properties of the configuration). The sparse sampling reduces the number of available allocations for these heavily oversampled objects, but does not remove the object from the possible allocations. Note that at this time the effect of this sparse sampling on properties such as spatial clustering is unknown. In most cases a better construction of the .fld file, with serious thought given the the true requirements of the project, is more appropriate than using sparse sampling on a poorly defined input file. To use the sparse sampling, set the seed for the random number generator, and then set the percentage of allocations to sample. Using only 10% will result in a very quick configuration, but most likely a poor yield. Using 80% seems to give a significant improvement in speed, without an obvious detrimental effects on the yield. Note: this mode is still underdevelopment, and it's effects are poorly understood at this time.

    Expert Allocation menu
    Figure 4: Expert Allocation options in Configure

     

    Setting up complex configurations

    Some of the strategies used for complex configurations that have been adopted in the past are outlined here. This is not an exhaustive list and we welcome comments and suggestions.

    Large scale survey programs have their own special requirements with regard to target allocation priorities. The GAMA survey project (Driver et al. 2009 A&G 50 12) implemented a very detailed, multi-year, survey strategy which is documented in Robotham et al. 2010 PASA 27 76. While this strategy is likely more complex than most programs will need, many of the issues of concern are discussed.

    The problem

    The 2dF positioner has only ~350-400 fibres once sky fibres and the current status of fibres is taken into account. Therefore, the ideal observing program has of the order of 400 targets per 2degree diameter field and a uniform target distribution with no closely spaced targets. Unfortunately, nature does not work like that. This results in extended target lists, with multiple priority levels and strongly clustered source distributions. The Configure software is necessarily generic, requiring the observer to carefully define input target lists.

    The main points to consider are:

    • Number of targets. In most cases, and particularly so for the default Simulated Annealing Configure algorithm, simply passing a list of >1,500 targets to Configure will not produce optimal results. With around 350-400 fibres available per configuration, target lists should normally be re-sampled to include only a small excess of targets to fibres in the input file. Around 800 targets works well for relatively uniform fields, lower numbers are needed for more compact fields.

    • Repeat observations. In surveys where the same (or overlapping) fields will be targeted multiple times, it is often advantageous to reallocate targets between observations. This can increase the total target yield by rejecting targets that are confirm to have unwanted spectral types, or replacing objects for which the spectral quality obtained is already sufficient.

    • Locking a sub-set of allocations. You may wish to force the repeat observation of a sub-set of high priority targets, at the expense of a large number of lower ones, but also include these lower priority targets in a new configuration where possible. There is a facility to lock fibres in place between configurations.

    A simple way to cover multiple targets in a field, using repeat observations, is outlined in section 5.4 of the configure manual "Multiple configurations to cover a target list". The process uses the option to save the unallocated targets from a configuration to a file, using the menu option File->list... in Configure. While this technique certainly works, and is discussed below, it is limited in that it can be rather tedious to undertake, and also cannot be done efficiently in advance of a run, since the 2dF astrometric solution will change before the run and so many allocations will not be valid during the obseravtions. A better solution is outlined in Example 2 below.

    Example 1 - Simple multiple configurations

    Starting from a large master catalogue:

    1. Allocate the field using configure

    2. Save the .sds file.

    3. Also save a .lis file (File->List...) which contains the unallocated objects (this is an option in the pop-up that will appear then you select File->List...). The .lis file format for the unallocated objects is the same as that for an input .fld file. If you select one of the other save options, the formatting will be a little different.

    4. Load this .lis file into configure and rerun the configuration, saving a new .lis file of the outstanding unallocated objects each time.

    Example 2 - Configuration using a master catalogue, and current status list

    For the reasons below, Example 1 above is not the preferred approach for most programs:

    1. For the first run in any 2dF-fed observing, the astrometric files will not be available prior to the run, and hence the configurations would be invalid if prepared in advance. This is typically only a small effect, but is compounded with each new iteration.

    2. The fibre availability of the field plates will change with time during a run, due to the slow rate of fibre attrition.

    3. No accounting is made for data quality once observations have begun.

    4. There is little flexibility between the interchange of the two 2dF field plates during observations.

    The solution is to prepare a master catalogue, with target observation priorities, for your full observing region and to create a processing script to draws sources from this catalogue based on external constraints. For example:

    1. Choose the first pointing centre for observation.

    2. Make a .fld file, select 500-800 high priority targets from the master catalogue. The UNIX grep and awk commands are ideal here, or a simple Perl script may be the best way to achieve this goal.

    3. If there are a small number of high priority targets, pad out the .fld file with lower priority objects. Care should be taken if a large number of low priority targets are introduced. The user should examine the configuration at step 5 below, to ensure a sensible fibre distribution is being used.

    4. Insert ~30-50 guide stars and ~50-100 blank sky positions.

    5. Configure and inspect the configuration

    6. Observe the field.

    7. Examine the spectra (determine redshifts, measure radial velocities, classify objects etc.)

    8. Update the master catalogue to reflect these observations. A safer approach is often to create a second catalogue of new target priorities. The processing script would then draw targets for subsequent configurations from the master file, but the priorities of classified targets are adjusted based on the information you have entered into the new target priorities catalogue. For example: Set the priority of objects with satisfactory spectra to the lowest value (Pri=1); Remove objects of the wrong spectral type; Flag objects with promising spectra, but which will need higher signal-to-noise. Note, you clearly cannot make the new target priorities until you have some data, and since one needs to have a number of .fld files ready in advance of each nights observing, this process is most efficient if any given region can be broken down into a number of independent (non-overlapping) pointing centres each night.

    9. Note, that there is no underlying reason why the field centre need be identical to that previously used.

    10. Repeat steps 5-9 until the required target completeness is achieved.

    Locking Fibres

    Please note: In the discussion that follows, it is assumed that you are trying to lock the allocation to the same field plate for which it was originally configured, and that you are observing at a similar Hour Angle (HA). Trying to lock the fibres to the same targets on different field plates or for a very different HA may fail to configure due to fibre collisions.

    It can often be useful to lock a number of fibres (any number between 1 and 399) onto certain targets while still allowing Configure to freely allocate the rest of the fibres. The classic example is that a field has been observed for 2 hours on one night and it has returned redshifts for half of the targets but the remaining targets need to be observed for a second 2 hours, as per the original telescope proposal, while adding in additional targets for the remaining fibres.

    It is possible to force the allocation of some fibres on to previously observed targets, and then reconfigure the remaining objects.

    The procedure for creating and using an import file

    1. Load your .fld file as normal, and configure as normal.

    2. Save the .sds file.

    3. Save a .lis file using the File->List.. menu option.

    4. Edit the list file to create a new file, by default Configure is expecting a .imp file. The nature of the Edit is discussed below.

    5. Reopen your .fld file (NOT your .sds file).

    6. Turn on Expert mode in Configure (select the Expert flag in the Options menu).

    7. From the Commands menu, select Import Allocations... and select your .imp file. This will allocate the fibres as specified in the .imp file.

    8. Now select Lock Allocations.

    9. You can now proceed with the normal configuration, as you did in step 1 above, but the locked fibres will stay in place.

    Creating your .imp file

    The file saved by the File->List option in Configure, a .lis file by default, is a plain text file which shows which fibres are allocated to which targets. The format is close to that of a .fld file, but with an additional column of * and a column of Fibre numbers between the * and the Object Names.

    The .imp file format is almost identical to that of a standard .fld file, but with the addition of an extra column of fibre numbers. This column should be the very first column, i.e. it goes before the Object Name column of a standard .fld file.

    To convert the .lis file into a .imp file, simply delete the first column of * characters that have been added.

    Modify the contents of the .imp file to only list those fibres needing to be locked, or else all 400 fibres will stay locked. For some programs it may be possible to edit the .imp file by hand in a text editor. For most programs, you will probably want to write a simple script to remove or re-prioritize allocations based on the results of an initial examination of the spectra from a first observation.