The COS pipeline propagates both statistical errors and data quality flags
throughout the calibration process. These are then combined from both the science data and the reference file data to produce data quality information in the calibrated data.
The error array contains an estimate of the statistical error at each pixel. In the raw
file, the error array is empty, and in the calibrated files the error array is generated from Poisson statistics of the gross counts and propagation of the errors to account for those introduced by correcting the observed data for flat-field and dead-time effects.
Every photon event in a COS corrtag file has a Data Quality (DQ) flag (Table 2.19
). Each flagged condition sets a specific bit in the 16-bit DQ word, thus allowing each event during an exposure to be flagged with multiple conditions using the bitwise logical OR operation. DQ flags can be divided into four types:
flags mark events which fall on a detector region which may be questionable. The BPIXTAB reference file marks the corners of each region on the detector which falls into each of these categories. Separate BPIXTAB files are used for the FUV and NUV detectors. These regions were determined by visual inspection of a set of science data files. For FUV data, the GSAGTAB is applied along with the BPIXTAB. The GSAGTAB is used to flag regions that are severely gain sagged
The DQICORR step of calcos
maps these spatial regions to the individual photon events, and the x1dcorr
module uses these flags and the value of SDQFLAGS to create the DQ and DQ_WGT arrays, and ultimately which events to include in the final (x1dsum
) spectrum (Section 3.4.15
). For more information on DQFLAGS and SDQFLAGS consult http://www.stsci.edu/hst/cos/pipeline/cos_dq_flags
flags mark photons that occur during time spans in which the data quality is suspect. Events flagged in this way will be removed from the data products, and the exposure time will be adjusted accordingly. Two types of temporal flags are used:
flags are set by calcos
if a photon event falls outside defined thresholds. Currently, only the FUV Pulse Height flag (512) falls into this category. All FUV events with pulse heights falling outside the range specified in the PHATAB reference file will have this flag set, and the data will be excluded by the DQICORR module. This flag does not apply to NUV data.
4. Lost Data
flags occur if data are missing for some reason, such as errors in transmitting the data from the instrument to the ground. Data marked with these flags is always excluded from the final products. There are two flags in this category:
Screening for temporal and event flags is done by turning calibration switches on
or off, or by altering reference files. Once a photon has been determined to have a bad temporal or event flag, it will never appear in a final data product unless the modules which screen for it are turned off or the reference files which define them are changed. On the other hand, the screening for the spatial flags can be easily altered by changing the SDQFLAGS keyword in the header of the raw data file.
The DQ extension of raw ACCUM
files will be filled only when there are missing (data lost) or dubious (software error) data. If no such errors exist, initialization will produce an empty data quality extension whose header has NAXIS=0
. These flags are set and used during the course of calibration, and may likewise be interpreted and used by downstream analysis applications. See Section 3.4.13
for more information on the data quality initialization calibration module.