AAOmega, Configure and Simulated Annealing

A warning on the use of target Priorities

A successful SAconfigure strategy
New Configure options

Expert Configure options - sparse sampling for speed

Frequently made mistakes

Configuring a field of fibres for a 2dF observation is a complex procedure. The Configure software was designed to take the strain out of the process. Configure version 7 (and all previous version of configure) employed two configuration algorithms, the so called Oxford algorithm (developed primarily by Gavin Dalton, for the 2dFGRS) and the Taylor algorithm (developed by instrument designer Keith Taylor). The Oxford algorithm proved most popular with the majority of programs and was used by default in most cases. This algorithm is based on an allocation strategy akin to that used by a Chess computer, in the way that the consequences of individual allocations are assessed in an iterative process.

Given the computing power available at the start of the 2dF project and the goals of the 2dFGRS for which it was designed, the Oxford algorithm represented an excellent solution to this complex problem, and is still appropriate for many AAOmega projects since it is relatively fast, and simple to use. However, it suffers a number of limitations:

1. Weighting of target priorities is not optimal
2. The limited use of fibre swaps can result in poor yields
3. Non-uniform sampling that can imprint power onto target distributions and reduce area of field. This is particularly a problem for higher density fields.

With this in mind, Brent Miszalski & Keith Shortridge have implemented a Simulated Annealing (SA) algorithm within Configure, which aims to address these short coming, but at the expense of an often increased configuration time, and a little more effort at start up from the user. The advantages of SA configure are explored thoroughly in Miszalski, Shortridge and Saunders et al. (2006 MNRAS 371 1537), (arXiv:asto-ph/0607125), and are summarized in Brents article in the February 06 AAO newsletter. Those interested in the concept behind Simulated Annealing might wish to start with the discussion in "Numerical Recipes" (Press, Teukolsky Vetterling and Flannery section 10.9). In short, SAconfigure provides

Increased yields for complex fields
Optimal target priority weighting
Highly uniform and quantified sampling behaviour
High completeness for fibre pairs when allocating a field for Cross Beam-Switching observations

Limitations of SAconfigure

In order to perform the large number of fiber swaps required by the SA algorithm, the first step in running SAconfigure is for the software to generate a fibre collision matrix. This matrix specifies the allowed configurations of each target/fibres, given a particular fibre+target allocation. For a uniform target density, this matrix is of order NxN, where N is the number of input targets (science targets, calibration sources, fiducial stars and sky fibres. For a compact or highly nonuniform field, the size of this matrix grows much more rapidly with N. The time required to generation of the collision matrix is therefore proportional to:

The number of targets in the input .fld file.
The density of targets in the input .fld file.

For a relatively uniform target distribution, such as the level of clustering seen in an individual 2dFGRS field configuration, and 500-800 targets, the time required for an SA configuration can be 20mins on a modern LINUX machine. For a similar field, with 1,500 targets, or for a centrally condensed field such as a galaxy cluster, the SAconfigure time could be 1-2hours! Therefore the user will be required to perform some kind of sparse sampling of the data set in order to run SAconfigure. There are no hard and fast rules as to how to achieve a good result, a strategy that has been successful in the past is outlined below. Contact your support astronomer for more details.

Note: the latest versions of Configure (v7.9 onwards) are multi-threaded and so multi-CPU machines will generate the collision matrix much faster, reducing the problems discussed here somewhat, but NOT removing them.

A warning on the use of target Priorities
The SA algorithm is very good at allocating targets based on priorities. However, the user should exercise some restraint when using the 9 available priority levels. While it is inviting to use all of the available priorities and to derive a complex priority selection function, this will almost always yield very limited returns at the expense of usability (which will provoke more errors by an inexperainced user, potentialy compromising the program as a whole). For most programs the number of targets in a given .fld file must be restricted (as described below) in order to allow the configuration process to be completed in an appropriate amount of time (20mins is perhaps a good guide here). A field that is simply stacked with a large number of low priority targets will take a long time to configure, even when using a fast multi-processor machine. If these targets are indeed low priority then the user should consider carefully whether there inclusion in the .fld file is worth the overhead in configuration time they will incur. If the answer is yes, then one might argue that their priority is in fact higher and they can be sampled along with other targets as described below. If the answer is no, then simply remove the targets. If the much quicker configuration shows unallocated fibres, then one might reconsider this question. As with all things the inclusion of low priority objects into a .fld file is a trade off between usability and end result. The inexperienced user will often achieve better results with less complex input file which allow rapid alterations to the configuration as the observing program progresses. Ultimatly, the user should discuss the situation with the support astronomer and try a few itterations.

A successful SAconfigure strategy

This simple recipe can result in high yields of high priority targets in even the most complex of 2dF fields.

The total target number is composed of:

All Science targets
All fiducial stars
All sky positions
Any calibration sources

Note: if you need some calibration sources in a given configuration, the only way to achieve this is to add a small number of such calibrators, and set them to priority P=9. One should then shuffle the priority of all science targets to lower levels, the highest priority targets having P=8, so that the calibrators are always allocated. If you find that you have run out of priority sub-devisions because of this then the chances are that you have not properly specified your target list (see the section above).

Typical numbers for each type of source might be:

Raw source catalogue
Sky positions	100
Fiducial stars	50
Calibration sources	0 to 10
Science targets	2 to >5,000*

Note:* Even with a rather powerful PC, with multiple CPUs, and a lot of free time, SAconfigure is NEVER going to be able to handle an input .fld file with in excess of 1,500 sources in it. Some level of sub-sampling WILL be required for fields with high target numbers.

A working maximum for a modern Linux PC seems to be around 500-800 targets per configuration file, depending on target density in the field. For a field with all it's sources in the central square degree, this may need to fall to <400.

When preparing the .fld file for SAconfigure the following steps should be followed

Select a uniform sample of fiducial stars, ~50 seems to work well.
Selection suitable blank sky fibre positions, ~100 typically works well.
Select a number of calibration sources (if required), these should be set to Priority 9 in the .fld file.
Select N High priority (P9) sources, from your input target list, where N=Ntotal - Nfid - Nsky - Ncalib. For uniform target densities, Ntot=800 has been seen to work well. For highly non uniform configuration, 500 or 600 may be more appropriate. Be sure to random sample the input list, if it was RA or Dec ordered then the output .fld file may not cover the full field plate.
If there are fewer than N high priority targets in your input list, then proceed down to the next priority level (P8-to-P1) until the desired value of N is reached.
Start up SAconfigure and run the configuration. If the time required for the generation of collision matrix is too long (and most support astronomers tend to lose patients after around 20minutes) then repeat the above procedure with a smaller value of N

New Configure options

There are a number of new configure option available with the SAconfigure algorithm. A number of these options can only be accessed via the Expert user mode which one activates via the toggle setting in the Options menu. Take care with Expert options, they can do harm to your science if you do not fully understand their implications. Discuss the issues with your support astronomer well in advance of your run if required.

Annealing. The rigor of the annealing process can be adjusted to meet each projects requirements. This governs how quickly the annealing routine cools during the allocation process. In practise, we have found that the Standard setting is almost always adequate.
Weight close pairs: ThetaMin; ThetaMax. In some circumstances one may wise to give additional weight to closely packed targets, at the expense of overall target yield. These objects allow this to be setup, but beware of the odd effects it will have on your allocation. This option has not bee extensively tested to date.
Cross beam switching. If the observation requires Cross Beam Switching (CBS) between pairs of fibres, then the user should first generate the paired target positions using the menu option Commands->Generate CBS pairs and then set the CrossBeamSwitching flag. This gives additional weight to target which are successfully allocated pairs of fibres, at the expense of overall target yield.
Straighten fibres. This gives increased weight to allocations which have fewer fibre crossovers. While this will have some impact of target yield, the effect is small/undetectable for most source distributions when using the Moderate setting and results in fields that typically requirer fewer fibre parks between configurations, hence reconfiguration is faster (perhaps by 10minutes, ~20%, in some cases). Fig. 13 of Miszalski et al. 2006 shows the effects of this straightening. It can have adverse effects on target priorities and so the concerned user will need to experiment with this option to determine the optimal solution.
Collision Matrix. It is occasionally useful to save the matrix of fibre collisions which has been calculated for this field. This enables quick restarts of the software later on. This file can however be rather large.
Enforce sky quota. This option forces the allocation of the requested number of sky fibres. This can result in subtly lower target yields for some fields, although the effect is small/undetectable for most source distributions (accepting that the full sky quote is allocated to skies). Most datasets will be of little value with less than 15 sky fibres. 20-30 fibres is more typical for most projects.
Peripheral weighting for Fiducials. This gives enhanced weight to selection of stars towards the edge of the field, which is typically beneficial for acquisition, and prevents all of the fiducial stars being crowded into a small area of the plate, as can happen with the SAconfigure algorithm.
Weight fiducial target pairs. For CBS observations one may wish to allocate the fiducial fibres in pairs in order to guide in both positions of the beam switch. Setting this flag gives extra weight to paired fiducial allocation. Note: it is often more efficient in terms of fibre allocation for the user to allocate fiducials by hand but to ensure that half of the fibres (e.g. 50, 150, 250 and 350) go to position A guide stars, while the other half (e.g. 100, 200, 300 and 400) go to position B guide stars. There is no requirement that these stars be the same set in the A and B positions.
Number of background threads to use. The calculation of the fibre collision matrix is very CPU intensive. On a modern multi CPU machine Configure can hijack all of the available CPUs and run a number of background threads, this vastly reduces the allocation time. For a single CPU machine, there is nothing to gain here.
On-the-fly collision calculation. By default, the the collision matrix is calculated in full in advance of the annealing (this is the way Configure-v7.4 operated when SAconfigure was first introduced). An alternative is to calculate it on-the-fly. This ensure that A configuration is achieved as quickly as possible. This configuration will be HIGHLY sub-optimal. The longer the process is allowed to run, the greater the region of parameter space that is investigate and the the better configuration will be. In the limit of the annealing process, the two approaches will produce identically good configurations, and will take identically long to reach this point. There is therefore often little point in doing the calculations on-the-fly. In fact this option may allow inexperienced/inpatient users to produce sub-optimal configurations. It can however, be used in cases where a pretty good configuration is needed rapidly. Note: the original Oxford configuration algorithm, which can be used instead of the annealing by running the configureTrad command, will be far quicker.
Number of Sky fibres. The indicated number of sky fibres will be assigned (but see the note above on enforcing the sky fibre quota).

Additional Expert alloaction options

Fibre clearance, Button clearance and pivot angle. These options are mainly for the 2dF support staff. If you do not know what they are used for then you should not adjust them. Note that the 2dF robot has safe values HARD WIRED into the system and so a configuration which is outside these bounds will be flagged as INVALID at configuration time. These settings should only be used to restrict the values to tighter constraints for reasons that are beyond the scope of this web page.
Random Seed and Percentage of allocations sampled. If one needs to configure more quickly, e.g. if the field is pathologically complex (usually centrally condensed or with heavily clustered targets) and one cannot alter fibre allocations as described above (and on the complex configurations page), then it is possible to sparse sample the collision matrix and speed up the process. The details of this option are beyond the scope of this web page and should be discussed with your support astronomer. The principle is, for such configurations, that the slow speed is caused by the large number of rather similar configuration that are available (in essence many objects could be configured with many different fibres without changing the basic properties of the configuration). The sparse sampling reduces the number of available allocations for these heavily oversampled objects, but does not remove the object from the possible allocations. Note that at this time the effect of this sparse sampling on properties such as spatial clustering is unknown. In most cases a better construction of the .fld file, with serious thought given the the true requirements of the project, is more appropriate than using sparse sampling on a poorly defined input file. To use the sparse sampling, set the seed for the random number generator, and then set the percentage of allocations to sample. Using only 10% will result in a very quick configuration, but most likely a poor yield. Using 80% seems to give a significant improvement in speed, without an obvious detrimental effects on the yield. Note: this mode is still underdevelopment, and it's effects are poorly understood at this time.

Basic allocation options	Expert allocation options

Frequently made mistakes

For a very densely populated field, the time taken to generate the collision matrix is a stronger function of N that NxN. In this regime, simply reducing the number of targets in the input file will not necessarily increase the speed of the collision matrix generation significantly. In this case one should sparse sample the very dense regions of the field. The total number of targets in the input file can remain quite large, provided the numbers of targets within the high density regions is small. Since the minimum fibre separation is of the order of 30arc seconds, there is little to be gained, even with SAconfigure, on having a high target degeneracy on scales smaller than 0.5-1arcminuates.
Do not allow multiple instances of a target to appear in any given .fld file. There is a large penalty for this.
Check the selected fiducial stars. Unless you have selected the Weight Peripheral Fiducials option, the SAconfigure algorithm can pick some rather poor guide star selections (e.g. all stars in the center). Do check to make sure the arrangement is sensible (i.e. well separated across the field plate)
Check the number of sky fibres allocated. Unless you have selected the Enforce Sky Quota option SAconfigure can some times leave very few sky fibres, if any. This might invalidate your whole observation, and so it is important that the number of sky fibres is sufficient.

Rob Sharp (rgs@aao.gov.au)