F I T S _ H E A D E R _ E D I T O R ( F H E ) S.Lee 1995/09/12 v1.2 1998/02/14 FHE is a program to allow the header portion of a standard FITS file to be examined and edited. Existing keyword's values may be changed or deleted, and new keywords may be added. It supports multiple COMMENT and HISTORY lines; which may be edited, deleted or inserted. Values may be logical, numeric or character strings. I consider FHE to be a very complex program because its aim is to allow simple editing commands to operate on a header whose structural integrity MUST be maintained. The data entered are also bound by many rules. Whilst I have extensively tested this program and believe it to work reliably, I would not be at all surprised if some condition caused it to fail - in which case I would like to be told. BACKGROUND INFORMATION: ----------------------- It's a good idea to understand a little about the format of a FITS file and the header so that you know why certain things are possible and others are not. FITS stands for Flexible Image Transport System, and rather remarkably seems to have born out its name. It has become the de facto standard for storing and transporting astronomical image files. It was originally written as a format that would allow easy transport of data between the originating source (the telescope) and the observer's home institution where the data would be examined. It was expected that the data would initially be recorded in whatever format was convenient for the instrument and would be examined using whatever software the observer was familiar with. All that needed to be written was a to-FITS converter at the telescope and a from-FITS reader at the other end and all data taken at any telescope could be examined at any institution. Data are stored as either 8-bit unsigned integers, 16- or 32-bit signed integers (2's complement), or 32- or 64-bit floating point. Integers are 'little-endian' i.e. bytes of decreasing significance with the sign bit first and the 1 bit last (this means byte-swapping for the IBM PC architecture). Floating point numbers must conform to the IEEE-754 standard for 32-bit and 64-bit floating point numbers. (FITS does not directly support 16-bit unsigned integers that many CCD systems produce. However, valid FITS files can be produced from such data by setting the BZERO keyword in the FITS header equal to 32768 and the BSCALE keyword equal to 1. A FITS reader will then add 32768 to the value in the file, restoring the original value, before interpreting it.) The key to its flexibility is the header which allows any configuration of data to be defined and also to hold any amount of additional useful information about the data. The header is readable by both computers and humans - an important consideration. The other aspect, important when defined but less so now, is the block structure of the file. The physical medium for transporting the data was a magnetic tape. These were typically written in blocks - chunks of data of a fixed size. The size of a FITS block is 23040 bits. Why this strange number? Simply because it is a multiple of all the "word" sizes of the computers in use at the time. This means that the block could be written with a whole number of "words" from any computer regardless of word size. These days this factor is unimportant as the "byte" - 8 bits - has become the standard, but when the FITS standard was defined computers had word sizes of 6,8,12,16,18,24,32,36,48,60 and 64 bits but they could all be accomodated with 23040 bits. This block leaves one legacy - the header. The header is one or more complete blocks - the data starts at the beginning of a new block. The other historical artefact is that the header is broken into 36 lines each of 80 characters. Eighty characters is the size of a computer punch card, the standard method of inputting data into a computer at that time. The 80 character line is broken into an 8 character keyword (which must be in upper case) and the data which it represents. An optional comment can fill the rest of the line. The format is only slightly flexible. o Columns 1 - 8 are the keyword. Only certain characters are allowed. o Column 9 is the '=' character (except with 3 specific keywords). o Column 10 is a space (except as above) o Columns 11 - 30 (and sometimes 31 - 50) are the data, but may be extended provided certain conditions are met. o The rest of the line is a comment. The data in columns 11 - 30 are either numeric, logical or character. The data values follow the rules of ANSI FORTRAN 77, although it is usually the simplest format that is used. - If the value field is numeric, the value is right justified to column 30. Values may be integer or real. If real, the decimal point is required. Exponential notation is allowed. (Imaginary numbers are allowed for, too, right justified in columns 31 - 50, but need not concern us here.) - Logical values are either the character T (for true) or F (for false) in column 30. - Character strings are enclosed between single quote marks. The leading quote is in column 11 while the closing quote is is anywhere after column 20 - i.e. 8 characters are the minimum length of a string. If the string is less than 8 characters it is padded with spaces, if the string is longer it carries on (provided that a closing quote terminates the string). Normally, if a string is 8 characters or less then the closing quote would go in column 20, but FHE puts the closing quote in column 30 so that the header looks tidier - at least I think so. - The remainder of the line is a comment, although it is usual to separate it from the data by a forward slash '/' character. The strict FITS rules require that the data values be in upper case, but this is often relaxed these days. (Keywords must be in upper case, however.) There are 5 specific keywords that must exist and they must be the first keywords in the header. Anything else is optional. These keywords are: - SIMPLE This is a logical flag which indicates that it is a 'simple' FITS file. The character 'T' appears in column 30. - BITPIX The number of bits per pixel. Either 8, 16, 32 for integer data, -32 or -64 for floating point data. - NAXIS The number of axes in the data. The value of this parameter determines how many other keywords must appear. (NAXIS may be 0, in which case no NAXISn records appear, nor (probably) any data. - NAXISn The dimension of each axis. The n is a number between 1 and 999. There must be as many NAXISn keywords as are stated in the NAXIS data. For our usual 2 dimensional CCD data, NAXIS is 2 and there will be a NAXIS1 and NAXIS2 keywords. The first axis (NAXIS1) is the one whose values increment fastest in the array. - END The last keyword in the header. Any remaining space within the block is padded with the space character. (Any other keywords in the header go between the NAXISn keywords and the END.) These few keywords are all that is required to make a valid FITS file. The FITS definition also details other keywords that may be used and suggests possible others. One of the complaints against the FITS file definition is that it is too free - keywords may be created by anyone and used how they see fit. However, there are enough recommended keywords to keep us going and they are generally used in a standard way. Some keywords which may be present in a header are: Keyword type purpose ----------------------------------------------------------------------------- BSCALE numeric Scaling factor for data. BZERO numeric Offset value in data. Used in conjunction with BSCALE to increase the dynamic range of integer data. value = (tape * BSCALE) + BZERO OBJECT character Image name. OBSERVER character Name of observer. INSTRUME character Data acquisition instrument. TELESCOP character Data acquisition telescope. ORIGIN character Institution of original data DATE-OBS character UT Date of data acquisition (DD/MM/YY). DATE character UT Date file written in DD/MM/YY format COMMENT character Any comments. Does not need '=' in column 9. HISTORY character Comments, usually about processing data. (blank) character More comments, but often just a completely blank line put in to separate items in the header. CDELTn numeric Increment in physical coordinate along axis n. CTYPEn character Type of physical coordinate of axis n. DATAMAX numeric Maximum value in data array. DATAMIN numeric Minimum value in data array. There are a few other keywords which were not included in the original definition but are now understood. TIME-OBS character UT Time of data acquisition (HH:MM:SS). EXPOSURE numeric Exposure time. EXPTIME numeric IRAF uses EXPTIME for exposure. (Note: because of the short-sighted format of the DATExxxx keywords, a new standard is now being used. By definition, if the DD/MM/YY format is encountered, it is assumed that the century is 19. The new format uses a subset of the ISO-8601 date format, specifically yyyy-mm-dd and if time is included it becomes yyyy-mm-ddThh:mm:ss.s. (note the 'T' between the date and time fields). Dates and times are UT (or UTC) unless the TIMESYS keyword specifies something else.) There are many other keywords than these few listed here, but these are the ones that amateurs are most likely to come across. It is also recommended that all units be in the SI MKS system. i.e. Temperature in degrees Kelvin, time in seconds, angles in degrees, distances in metres and mass in kilograms. SUMMARY: -------- As far as we are concerned, we need to know that the header is composed of 80 character records, 36 to a "block". Header records are filled up in multiples of blocks. There are a certain number of mandatory keywords and many optional ones. The layout of each line is fixed, although slight variations are possible. Keywords may consist only of the characters A through Z, the numbers 0 to 9, or a hyphen or underscore. No embedded blanks are permitted. (It is also best not to start a keyword with a number.) When adding or removing items from the header, we must make sure that this block structure is maintained. If the changes made within the header don't alter the number of blocks, then the file can just be modified. But if a new block has to be added (or an existing one removed) then the whole file must be re-written to keep the structure valid, thus significantly complicating the program. USING FHE: ---------- FHE only allows 3 operations - adding a new keyword, deleting an existing keyword and replacing the value of an existing keyword. It can also list the contents of the header. FHE may be started by either naming the file to be edited on the command line, or allowing FHE to prompt for it. The file type defaults to '.fts' but can be overridden either by naming the file extension explicitly or by setting the environment variable FITS_EXT to whatever you want; e.g. SET FITS_EXT=FIT FHE first reads the header and prints out some statistics - how many lines there are, along with how many COMMENT, HISTORY and BLANK lines. It then requests either a command or a keyword. There are only 2 commands recognised; '?' and 'end' - anything else is treated as a keyword. Entering the '?' character will list the contents of the header, 20 lines at a time. Press the return key to see the next 20. You may quit the listing at this point by pressing the 'q' key before hitting return. Once you have finished with the program, type 'end' to write out the changed file and terminate the program. Nothing is written back to the file until the END command is issued. So if you've made a catastrophic mistake you can simply press control-C to crash out without modifying the file. Anything else entered is treated as a KEYWORD to be entered into the file. The validity of the keyword is check to make sure that it is 8 characters or less in length and contains no illegal characters. Note that all changes go in at the end of the header, there is no provision for moving lines around within the header or inserting lines at an place. Keywords may be entered in lower case, FHE will turn it into upper case before storing it. There is one keyword which can't be entered directly - the so called "BLANK" record where the keyword is 8 blanks. To enter a blank you must type in the code ".blank" (without the quotes). BLANK is a completely different and valid keyword. Blanks are usually put in just to make the header more readable to a human. FHE takes care of entering the "=" in the right place, putting quotes around character strings, lining up values in the right colums and separating the comment field from the value field with a ' / '. Lines are truncated to 80 characters. o If the keyword you enter does not already exist in the header, it will add it as the last keyword (suitably moving the END record). You will then be asked for the value. Keywords MUST have values. Next you are asked for a comment for the line. This is optional - just hit return if you don't want to comment it. o If the keyword already exists you then have two options - to delete it, or to modify it. FHE displays the current value of the keyword and then awaits input: - to delete it, just hit return; - to edit it, enter a new value. Sorry, it doesn't allow the existing line to be edited, a complete new one must be typed in. Keywords MUST have values. The old line's comment is retained. - Once you've entered a new value, you are asked if you wish to change the comment field. You must enter Y or N. If N(o) then no changes are made, if Y(es), you can then enter a new comment. Press return to delete the existing comment, or enter a new one. o There is a slight modification to the above if the keyword is either COMMENT, HISTORY or .blank and there is already one or more of them in the header. These keywords may have multiple entries in the header - no other keyword may. They also don't have comment fields. You are asked whether you want to enter a new record (at the end of header) or to edit an existing one. Enter 'o' or 'n' for old or new. - If a new one, then you just enter the value. - If you want to change an existing one, FHE prints out the existing ones, numbering them as it goes. You then enter the number of the one to edit. Again you have the option of deleting it or replacing it. Values come in 3 flavours - strings, numbers and logical. For the purpose of making FHE as automatic as possible, a simple scheme has been defined. o Logical values are strings consisting of only the single character 'T' or 'F'. (NOTE: you must enter this as an upper case T or F, lower case will be treated as a string.) FHE places it in column 30. (I think it unlikely that anyone would want a character string that was this format, but I'm sure that somebody will.) o Numbers are anything that contains only a valid numeric string, including scientific notation. This, too, is placed in the header, right justified to column 30. - The number string is stored exactly as typed, so you can control the format of numbers how you wish. However, the number string can only be 20 characters long, so if you enter more than that FHE will re- format the string using floating point notation (as a FORTRAN D20.13). - Imaginary numbers are not supported by FHE. o Therefore, a string is anything that doesn't fit into one of these two catagories. It is enclosed in quotes and placed into the header left justified from column 11. Strings may be up to 68 characters long - this leaves enough room for the opening and closing quote, but no room for any comments. It is unlikely that real values will be this long (nor should they be). Once you are satisfied with the changes to the header, type 'end' to write the new information to the file. Now one of two things will happen. If the changes made to the header did not alter the number of blocks necessary for the header, then the file is simply updated. But if either more or less blocks are now necessary, then the whole file must be re-written. This is accomplished by writing a temporary file, deleting the original, then renaming the temporary file. This naturally takes a little longer - it tells you what it is doing while it is doing it. EXAMPLE: -------- A sample header is shown below... ============================================================================ SIMPLE = T / Conforms to FITS standard BITPIX = 16 / Raw data:12 bits, unsigned NAXIS = 2 / 2 Dimensional data NAXIS1 = 378 / Pixels, 0.017mm NAXIS2 = 242 / Pixels, 0.01975mm BSCALE = 1.0 / Data not scaled BZERO = 0.0 / Data not scaled DATAMIN = .0 / Minimum data value DATAMAX = 2284.0 / Maximum data value CDELT1 = 0.017E-3 / Pixels, metres CTYPE1 = 'PIXELS ' CDELT2 = 0.01975E-3 / Pixels, metres CTYPE2 = 'PIXELS ' INSTRUME= 'Cookbook 245 CCD ' / Instrument DATE-OBS= '01/01/97 ' / day/month/year UT TIME-OBS= '14:22:58 ' / Start time HH:MM:SS UT EXPOSURE= 20 / Exposure in seconds EXPTIME = 20 / Exposure in seconds OBJECT = 'M42 - V ' / Name of object COMMENT M42 - V OBSERVER= 'Steven Lee ' / Name of observer TELESCOP= '20cm f/4.5+barlow ' / Telescope FILTER = 'V ' / Filter in beam ORIGIN = '"Rainbow" ' / Name of observatory OBS-LAT = -31.2786 / Latitude of observatory (S31d16'43") OBS-LONG= 149.2124 / Longitude of observatory (E149d12m44.5s) OBS-ALT = 580 / Altitude of observatory (580m) HISTORY Cookbook raw data converted to FITS format by CB2FITS v1.3 (S.Lee) HISTORY File (M42V-001.FTS) created on 1998/01/10 14:50:06 HISTORY The sum of 5, 20 second exposures; each dark subtracted and HISTORY flat fielded. Before summing, the images were grown by 2. COMMENT Taken as part of tri-colour sequence. END ============================================================================ FHE was used to insert the final 2 blank lines, 2 HISTORY records and final comment into the header. (The earlier blank lines and history records were placed there during file creation by CB2FITS.) LIMITATIONS AND CHANGES: ------------------------ FHE only makes room for 10 blocks of header records. That's 360 items, I think it should be enough. It also only allows 100 each of HISTORY, COMMENT and BLANK items. Again I think that's enough. If it isn't then let me know. There are a few features that should be add to FHE. Refusing to modify the mandatory keywords (on the assumption that they had to be correct in the first place) would be sensible. I'm open to suggestions...