Appendix C: Making Data Tapes at the End of your Run

C.1 Data Formats - Figaro, FITS & IRAF

Observers may take their data away in

• the FIGARO NDF format in which it was taken (i.e. as a .sdf file)
• in FITS format.
• in IRAF format
• or in any other format you choose to create from the FIGARO files created at the telescope.

When the data is created by the AAO instrument computers, it appears on the VAX disks in the following directories
 

These directories are also mirrored on the Suns via NFS-mounting.
 

Lastly, if you have been using the unix_server system with OBSERVER, the files you have created on the VAX will have been automatically copied to the Sparc station aatssf. These files will appear in the following directories.
 

Both RUN and DUMMY files will appear in the same directory. You should be careful with these files. unix_server copies files from the VAX to aatssf as a background process after the VAX has created the file. If you run out of space on /data/ssf/1/, unix_server will quietly fail. This means that it is possible for files to be missing from this directory. You can deal with this by (1) always trying to ensure the disk is not full (you can delete unwanted files - say the IRAF ones, if you only want FITS and NDF), and (2) re-copying the files from the VAX to the Sun at the end of the night before you make a tape. This is described below.

C.2 Making your own tape.

Tapes can be written in any format you like. On Solaris machines, all Exabyte tape drives are called /dev/rmt/1n, while DAT drives are called /dev/rmt/0m. Not all machines have both an Exabyte and DAT, but most machines can see the necessary discs.

Most tape drives at the AAO have the option of writing in a compressed and/or high density mode. Unless you have a lot of data and know that you have a corresponding type reader, it is recommended that you write your tape in uncompress, standard density; i.e. with the option lights off. A standard Exabyte will hold 2·3Gb of data, while a DAT will hold between 1·3 and 4Gb depending on the tape length.

First log into a Unix machine (e.g. aatssf) as obsred (your support astronomer will have given you the password).

C.2.1 Making a TAR tape from Figaro data on the VAX

This is a simple matter.

aatssf{obsred}: cd /vaxinst/ccd_1/970906
aatssf{obsred}: tar cvf /dev/rmt/1n *.sdf

Easy. But a bit slow, because the files get transferred over the network.

C.2.2 Making a TAR tape from Figaro, FITS or IRAF data already on the Sun

Lets suppose that you've either transferred the data to aatssf (see below) or are happy that unix_server has created all the files you want properly.

aatssf{obsred}: cd /data/ssf/1/obsred/ndf/ccd_1/970906
aatssf{obsred}: tar cvf /dev/rmt/1n *.sdf

Or for FITS files.

aatssf{obsred}: cd /data/ssf/1/obsred/fits/ccd_1/970906
aatssf{obsred}: tar cvf /dev/rmt/1n *.fits

Or for IRAF Files

aatssf{obsred}: cd /data/ssf/1/obsred/iraf/ccd_1/970906
aatssf{obsred}: tar cvf /dev/rmt/1n *.{imh,pix}

Again, easy. And faster than from the VAX.

C.2.3 Making a FITS tape from the VAX Figaro files

A FITS tape is a tape on which you sequentially store each image as a seperate file, delineated by end-of-file markers. It is the `standard' way to save FITS data, in the sense that you should be able to read such a tape on almost any machine. However, such a tape has the disadvantage that (1) you lose all file and directory name information when you write a file like this - all you know is that a given file is the nth on the tape, and (2) writing a FITS tape (an reading it later)  is significantly slower than making a tar archive of the same files.

(1)Figaro can write a FITS tape with the WIFITS command. First start Figaro, define the output tape device, then write the data. To write multiple directories, just repeat the loop command until done. Finally, remove the tape.

aatssf{obsred}: figaro
aatssf{obsred}: cd /vaxinst/ccd_1/970906
aatssf{obsred}: tapeo /dev/rmt/1n
aatssf{obsred}: foreach i ( *.sdf )
?      echo $i
?      wifits $i current \\
?      end

(2)Alternatively, you may make use of the way that Figaro can take its input from a file rather than just a single file name. First create a file containing all the files to be written (this may include reduced data as well as raw data), then use it as the input to the WIFITS command. Using the ls command and piping the output to a file is the easiest way to create the list, although you may need to edit this file to ensure the file names are in the correct format.

aatssf{obsred}: figaro
aatssf{obsred}: tapeo /dev/rmt/1n
aatssf{obsred}: ls /vaxinst/ccd_1/970906/*.sdf > files.lis
aatssf{obsred}: ls /vaxinst/ccd_1/970907/*.sdf >> files.lis
aatssf{obsred}: ls /vaxinst/ccd_1/970908/*.sdf >> files.lis
aatssf{obsred}: wifits @files.lis current \\

C.2.4 Making a FITS tape from the SUN FITS files

If you have the necessary FITS files already on the SUN, they may be written out as a FITS tape by the following method. Note that this recipe write the data with a block size of 28800. This is the standard way we write FITS at the AAO, though many other facilities will expect a block size of 2880. You may need to tell your FITS reader of this difference.

aatssf{obsred}: cd /data/ssf/1/obsred/fits/ccd_1/970906
aatssf{obsred}: foreach i ( *.fits )
?      echo $i
?      dd if=$i of=/dev/rmt/1n bs=28800
?      end

Remember to write a final tape mark before removing the tape.

aatssf{obsred}: mt -f /dev/rmt/1n eof
aatssf{obsred}: mt -f /dev/rmt/1n offline

C.2.5 Getting Files from the VAX Disks to the Sun Disks

(1)  There are two ways to get your data copied (or re-copied) from the VAX to the Sun. The simplest is to use the unix_force command in OBSERVER to force all data to be sent again. Simply enter under OBSERVER

Idle: unix_force all
 

An alternative is to do a Unix copy

aatssf{obsred}: cp /vaxinst/ccd_1/970906/*.sdf /data/ssf/1/obsred/ndf/ccd_1/970906

This assumes the directory /data/ssf/1/obsred/ndf/ccd_1/970906 exists. If it doesn't then you can create it  with the Unix mkdir command or stick the files in some other suitable directory.

(2) If you want to create (or re-create) your files as FITS images, then you need to either use unix_force with the FITS option turned on (the easiest), or use the ndf2fits command to convert all the files (more complicated).

aatssf{obsred}: figaro
aatssf{obsred}: cd /vaxinst/ccd_1/970906
aatssf{obsred}: foreach i ( *.sdf )
?      /star/bin/convert/ndf2fits $i:r /data/ssf/1/obsred/fits/ccd_1/970906/$i:r.fits bitpix=16 comp=A
?      end

This will create FITS files in your "fits" directory on the Sun, which you can save to tape as above. The setting of BITPIX will determine whether you write float (-32) or short integer (16) FITS files. This assumes the directory /data/ssf/1/obsred/fits/ccd_1/970906 exists. If it doesn't then you can create it  with the Unix mkdir command or stick the files in some other suitable directory.

(3) If you want to create (or re-create) your files as IRAF images, then you need to first start Figaro, and then convert all the files.

aatssf{obsred}: figaro
aatssf{obsred}: cd /vaxinst/ccd_1/970906
aatssf{obsred}: foreach i ( *.sdf )
?      /star/bin/convert/ndf2iraf $i:r /data/ssf/1/obsred/iraf/ccd_1/970906/$i:r bitpix=-32 comp=A 
?      end

This will create IRAF files in your "iraf" directory on the Sun, which you can save to tape as above. The setting of BITPIX will determine whether you write float (-32) or short integer (16) IRAF files. This assumes the directory /data/ssf/1/obsred/iraf/ccd_1/970906 exists. If it doesn't then you can create it  with the Unix mkdir command or stick the files in some other suitable directory.