Robotic AstronomyView this Special Issue
Research Article | Open Access
The Data Reduction Pipeline of the Hamburg Robotic Telescope
The fully automatic reduction pipeline for the blue channel of the HEROS spectrograph of the Hamburg Robotic Telescope (HRT) is presented. This pipeline is started automatically after finishing the night-time observations and calibrations. The pipeline includes all necessary procedures for a reliable and complete data reduction, that is, Bias, Dark, and Flat Field correction. Also the order definition, wavelength calibration, and data extraction are included. The final output is written in a fits-format and ready to use for the astronomer. The reduction pipeline is implemented in IDL and based on the IDL reduction package REDUCE written by Piskunov and Valenti (2002).
The HRT  was built by Halfmann Teleskoptechnik GmbH (Germany) and installed at Hamburg Observatory in 2005. This Cassegrain-Nasmyth F/8 type telescope has a 1.2 m aperture, an Alt/Az mounting, and final direct drives with high-precision absolute encoders. The only instrumentation of the HRT is the Heidelberg Extended Range Optical Spectrograph (HEROS). It is connected with the telescope via a polymicro FVP 50/70 fused silica fibre, equipped with microlenses on both sides of the fibre to adapt the HRT-HEROS F-ratios. At present, only the blue channel of HEROS (380–570 nm) is operating, while the red channel will start its operations in July 2009. The spectral resolution of the device is . The telescope is designated for long-time monitoring of active stars at its final location Guanajuato in Mexico.
2. Reduction Pipeline
The reduction pipeline is provided as fully automatic reduction pipeline including an automated wavelength calibration. It is started by the Central Control Software of the HRT system after the observations and calibrations have been finished. The reduced data are stored in an archive. This pipeline is implemented in IDL (Interactive Data Language) and uses the reduction package REDUCE, by Piskunov and Valenti . REDUCE is a powerful package providing the required functionality for the HEROS pipeline.
In the following the main reduction steps of the pipeline are described and these are represented in flow charts.
2.1. Preparation of the Reduction
Before the main reduction starts, it is necessary to prepare the reduction. The flow chart in Figure 1 shows the main steps of this part.
The first step of the preparation is the definition of the directory holding the raw data. Also a temporary directory and the directory are created, where results are saved on disk.
Thereafter the parameters and file names for reduction procedure are defined. All reduction parameters are supplied with the same values to all routines called by the pipeline at the different steps of the reduction. This is made for consistency in the reduction. The parameter values are read out from the parameter file for the blue channel.
2.2. Building the Master Calibration Images
The HRT system takes several calibration images: Bias, dark, and flat fields, before and after the observations. The respective images are averaged and then used as master calibration images. The main steps to build the master image are similar for bias, dark, and flat field. The flow chart in Figure 2 shows the main steps in this part.
The first step is a check of the variation in the calibration images. The percentage of relative variation between the arithmetic mean of the single images and the median of this values is calculated as
For the calculation of for the darks the single images are corrected by bias and for the for the flats by bias and dark. The percentage of relative variation is plotted in a log file.
The calibration images are split in two lists. If the same number of images is taken before and after the observation, then in the first list the images taken at the start of the observation are collected and in the second list those taken after the observation. Then, the percentage of relative variations of the images are checked for both lists. If the percentage of relative variation is higher than a threshold, the corresponding image is not used for building the master calibration image. Normally the total number of images in both lists after the check of is greater or equals 3. The standard deviations of the percentage of relative variation of images, Equation (1), are calculated for both lists and used as the thresholds to build the master calibration image. If the standard deviation is less than 3.5, then it is reset to the default value 3.5. This is the minimum threshold to find outliers in the images.
If the total number of images in one list is less than 3, an error message is obtained and the images are collected in a new list. The content of this new list is checked. Now there are 3 possibilities.(1)The total number of images is 3. The reduction ends, because one needs at least 3 calibration images to build a master calibration image.(2)The total number of images is 6. The new list is not split, because in the list must be at least 3 images.(3)The total number of images is 6. The new list is split in two lists.
Thereafter the single calibration images are averaged. As next step it is checked if an error arose during the combination of the single calibration images. In this case the pipeline stops and an error message is written to a file.
The master bias is subtracted from the average dark and flat field and additionally the flat field is corrected from the dark contribution. The dark correction is not performed, if the arithmetic mean of master dark is smaller than a predefined threshold for the dark correction, because in this case a dark contribution is negligible. After the subtraction of the master bias, the dark is time normalised.
The results are saved as master calibration images (hereafter, bias, dark, and flat field). To monitor long-term changes, the arithmetic mean of the single images and the corresponding standard deviation are saved in a log file. Also the standard deviation of both lists and the arithmetic mean, the median, and the standard deviation of the master calibration images are saved.
2.3. Order Definition
The flat field is used for the order definition. The central positions of the individual spectral orders are located and defined in this step of the reduction pipeline. If a single order is not found, then the pipeline stops and an error message is written to a file. The results of the order definition are saved in the order definition file. The flow chart in Figure 3 shows the main steps in this reduction part.
2.4. Blaze Extraction
During the next step in the reduction pipeline the spectrum of the flat field lamp is extracted. The flow chart in Figure 4 shows the main steps in this part of the pipeline. This spectrum (hereafter, blaze) can be used as blaze function. The blaze is used to eliminate the blaze function in the science spectrum and to correct for the difference in quantum efficiency of the pixels (Section 2.6). After the background correction the blaze is extracted like a science spectrum. The counts in the blaze are converted in electrons and the blaze is normalised by the exposure time. Finally the blaze is saved in the order definition file.
2.5. Wavelength Calibration
The spectrum of the ThAr lamp is used for the wavelength calibration. One ThAr image will be taken before and one after the observations. The flow chart in Figure 5 shows the main steps in this part. The flow of the wavelength calibration can be split in two parts: the spectrum extraction and the new wavelength solution.
ThAr Spectrum Extraction
The two ThAr spectra are reduced consecutively. The bias is subtracted from the ThAr image. If the arithmetic mean of dark is above the threshold for dark subtraction, then the dark is also subtracted from the ThAr image. Thereafter the ThAr spectrum is extracted and saved on disk.
For the automatic wavelength calibration a reference ThAr spectrum with 1D wavelength solutions of the several orders is used. The extracted spectra are compared with the reference spectrum. The shifts are calculated for each order via a cross correlation. After that the order shifts of the two ThAr spectra are averaged for each order.
The new 1D wavelength solutions of all orders are determined with the shifts and the 1D wavelength solution of the reference spectrum. Finally, the results, the shifts, and the spectral resolution of the reference ThAr spectrum are saved in a wavelength file.
To check the results, the pipeline creates plots with the residuals of the 1D wavelength solutions fits and a file containing the arithmetic mean and standard deviation of the shifts and the residuals of the 1D wavelength solution.
2.6. Spectrum Extraction
The final part of the pipeline is the extraction of the actual spectra. First, the spectra of all images are extracted. The images of the same object are coadded and then a summed spectrum is created.
Spectrum from the Single Science Image
The flow chart in Figure 6 shows the main steps in this part of the pipeline. The science images were collected in a list. The bias is subtracted from the science image, and if the arithmetic mean of dark is greater than a threshold for dark subtraction, the dark is also subtracted from the science image. If sky images from this image (object) exist, then the average of two sky images is created and subtracted from the science image. Next, a background correction is performed. After the background correction the pipeline identifies outliers in the science image and these are flagged. Thereafter, the spectrum is extracted, the counts in the spectrum are converted to electrons, and the spectrum is time normalised. In the following, the spectrum is divided by the blaze . This step eliminates the blaze function. Simultaneously the correction for the quantum efficiency is achieved. If the keyword noautocal is not set, then the wavelengths for the individual objects are corrected by the corresponding barycentric velocity shifts. Finally the spectra are saved in fits file.
Spectrum from the Coadded Science Images
This is the last part of the reduction pipeline. Here the spectrum from the summed science images of the same object is extracted. This reduction procedure is similar to the procedure of extraction the spectrum from the single science image, but additionally at first the images are coadded. The flow chart in Figure 7 shows the first steps in this reduction part. The rest is similar to the spectrum extraction a single science image.
The first difference between the both reduction steps is a check which objects have more than one exposure. These objects are collected in a list.
Another difference is the coaddition of the single images for the same object, after the bias and, if necessary, a dark and sky correction. The mean Julian date for the summed image and the barycentric velocity shift are computed.
The data reduction pipeline for the blue channel works fully automatically and stably. In case of an error in some positions in the reduction flow, a message is written in a file. Also log files are written during the reduction. With these error and log files the astronomer can check the reduction flow. In Figure 8 an example of the final output of the pipeline is shown. A spectrum of Alcaid taken with HEROS and extracted with MIDAS  is shown in Figure 9 for comparison only.
One problem in the reduction pipeline is the identification of faint outliers in the science image. Also the continuum normalisation process of late types stars may have problems in order to find the quasicontinuum segments of the spectrum.
A future task will be the creation of the data reduction pipeline for the red channel. In general this pipeline will have the same structure as the pipeline for the blue channel has.
Finally, the optimisation and regular support of both pipelines are important to obtain the best possible outputs.
- N. E. Piskunov and J. A. Valenti, “New algorithms for reducing cross-dispersed echelle spectra,” Astronomy and Astrophysics, vol. 385, no. 3, pp. 1095–1106, 2002.
- P. Škoda and Šlechta, “Reduction of spectra exposed by the fiber-fed echelle spectrograph HEROS,” ASCR, vol. 90, pp. 40–60, 2002.
Copyright © 2010 Marco Mittag et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.