Calibrated Frame Product¶
$SetSchemaPath vis/euc-vis-CalibratedFrame.xsd
Data product name¶
$PrintDataProductName
Data product custodian¶
$PrintDataProductCustodian
Name of the Schema file¶
$PrintSchemaFilename
Last Edited for DPDD Version¶
1.1
Processing Element(s) creating/using the data product¶
VIS_science_xml_out
Processing function using the data product¶
SHE and MER processing functions.
Proposed for inclusion in EAS/SAS¶
Yes
Data product elements¶
$PrintDataProductElements
Detailed description of the data product¶
The VIS processing function provides a Calibratedframe product consisting of 3 FITS files by exposure: a calibrated VIS individual exposure, and the corresponding Background map and Weight MAP.
A PSF file is also provided. This corresponds to the PSF calibrated on a data set and is the same for all frames.
Some data cannot be used for scientific purposes, or can only be used with certain precautions. This includes data affected by, for example, stray light, X-rays or guidance errors. For selected useful data, some metrics are included in the .xml file that follows each FITS file. See Vis Analysis Results Product for how to interpret these metrics.
Calibrated VIS individual exposure¶
The calibrated VIS individual exposure is a multi-extension FITS file containing 3 extensions per CCD, one for pixel data, one for the associated RMS map and one for the associated Flagmap, to make 108 extensions for a full 36 CCDs which constitutes an exposure. Pixel data is 32-bit floating point format.
Each extension contains the keyword EXTNAME, which will be <detector id>.SCI, <detector id>.RMS or <detector id>.FLG for Science, RMS map or Flagmap. The “detector id” describes the extension following the scheme given on VIS FPA ICD (EUCL-SAP-ICD-6-001 and Fig. 11).
Fig. 11 : CCD numbering and location at detector plane level (Front view) from VIS FPA ICD (EUCL-SAP-ICD-6-001) issue 2.5 2014/20/10¶
The different extensions may appear in any order. In all extensions, pixels are delivered at the native pixel scale and all extensions have the same pixel scale and size.
SCI extension¶
The SCI extension contains calibrated pixel data for each of the VIS instrument CCDs. The individual images contain both an astrometric and photometric solution written in the FITS header. No flux scaling is applied, but the results of the photometric solution computed by VIS is written in the FITS header keywords. The astrometric solution is described as a PV projection. Images are not resampled and are delivered at the native VIS pixel scale (this means of course that instrument optical distortion is still present). The images are not background subtracted but a background map is provided as a FITS file with the same pixel size as the individual images.
For the calculation of the astrometric solution, we use WCSFITS. It uses a distortion model of the FPA to compute the astrometrical solution (pointing and position angle). The pointing position is the centre of rotation of the FOV and is defined as the centre of the VIS FPA.
RMS extension¶
The RMS map contains the noise at each pixel in the corresponding Science image, expressed as the absolute standard deviation and are the ones to use for photometry.
Currently to meet the requirements set by other PFs, RMS maps are simply generated directly from the output image using the BACKGROUND_RMS SExtractor checkimage (Bertin & Arnouts 1996). These maps are generated by computing the standard deviation of pixels in a sliding window on the input images, after removing objects.
Note that these RMS maps only contain a noise contribution from the background, and do not contain a contribution from the Poisson noise of each individual source.
FLG extension¶
The Flagmap data format is defined in the data model and presented in Vis Flag Map Product in a human readable format.
Data Header¶
Primary Header |
|
|---|---|
SIMPLE |
File does conform to FITS standard |
BITPIX |
Number of bits per data pixel |
NAXIS |
Number of data axes |
FITS_DEF |
FITS definition name |
EXTEND |
FITS dataset may contain extensions |
FITS_DEF |
FITS definition name |
FITS_VER |
FITS definition version |
TIMESYS |
Time scale of the time-related keywords |
PLAN_ID |
32-bit SOC Operation ID |
BUNIT |
Pixel data unit, e.g. ‘electrons’ or ‘adu’ |
OBASW |
On board application software version: version. |
DATE |
UT date when this file was created |
SOFTVERS |
Version of the simulator or version of the LE1 |
EXPTIME |
Commanded integration time in seconds, e.g. 565.0 |
IMG_CAT |
Data product category: SCIENCE, CALIB, TECHNICA |
IMG_T1 |
values in {BIAS, DARK, FLAT, LINEARITY, CHARGE_INJ |
IMG_T2 |
values in {LAMP, SKY, DOME, OTHER…} from SEQU |
OBSMODE |
Observation Mode (WIDE, DEEP, CALIBRATION) |
OBSTYPE |
DP technique (IMAGE, SPECTROIMAGE) |
INSTRUME |
VIS instrument |
TELESCOP |
Telescope name: EUCLID |
OBT_STA1 |
Integer, number of seconds of OBT (START_TIME) |
OBT_STA2 |
Integer, number of milliseconds of OBT (START_TIME) |
DATE-OBS |
UTC Date of observation (start), e.g. ‘2014-03-15T09:30:09.313’ |
MJD-OBS |
Inst. seq. start time in MJD |
FIELDID |
Field Id |
OBS_ID |
OSS_obsid |
DITHOBS |
OSS_inObsId |
PTGID |
OSS_pointingID |
EXPNUM |
Exposure Id in dither |
TOTEXP |
Total number of Exposures in the specific Point |
INSTRMOD |
VIS intrument mode: Science or Manual |
COMPALGO |
Science TM compression mode: NoCompression, Co |
COMPPARA |
Science TM compression parameters: pixels/block |
RA |
OSS_attEqLon |
DEC |
OSS_attEqLat |
PA |
OSS_attEqPosAngle |
EQUINOX |
Standard FK5 (years), e.g. 2000. |
SEQID |
VIS sequences as defined in EUCL-IFS-ICD-6-002 |
DATE-END |
Inst. seq. end time in UTC |
OBT_END1 |
Integer, number of seconds of OBT (VIS sequence) |
OBT_END2 |
Integer, number of milliseconds of OBT (VIS seq) |
NOMCHAIN |
VDGT2771 |
VSTART |
Index of first readout line (starting at 0) |
VEND |
Index of last readout line + 1 |
ROECFGID |
ROE configuration table ID |
RSUSINID |
Table version of RSU SINE table used in the RSU |
RSUFRID |
Table version of RSU FREQ table used in the RSU |
RSUSFRID |
Table version of RSU short FREQ table used in t |
RSUCFSTA |
Shutter unit configuration status: CLOSED_LOOP, |
TP_STAT |
Trap pumping status: off, parallel, serial, mul |
CUSTATUS |
Calibration unit status: On or Off |
CI_STAT |
Charge injection status: On or Off |
IDLTIME |
Time (s) between end of shutter opening or end |
OBT_STA |
OBT at start time |
DATE-ENH |
UTC of LE1 Enhancement |
AUX_VERS |
LE1 Enh. (AUX) Processor name & version |
ELAT |
OSS_attEclipLat |
ELONG |
OSS_attEclipLon |
POS |
OSS_attEclipPosAngle |
ALPHA |
OSS_alpha |
BETA |
OSS_beta |
SAA |
OSS_saa |
PATCH_ID |
OSS_obsPatchId |
FOV_LAT1 |
OSS_fovLat1 |
FOV_LAT2 |
OSS_fovLat2 |
FOV_LAT3 |
OSS_fovLat3 |
FOV_LAT4 |
OSS_fovLat4 |
FOV_LON1 |
OSS_fovLon1 |
FOV_LON2 |
OSS_fovLon2 |
FOV_LON3 |
OSS_fovLon3 |
FOV_LON4 |
OSS_fovLon4 |
VS1T2043 |
VS1T2043 |
VS1T2050 |
VS1T2050 |
VS1T2020 |
VS1T2020 |
VS1T2021 |
VS1T2021 |
VS1T2028 |
VS1T2028 |
VS1T2029 |
VS1T2029 |
VS1T2036 |
VS1T2036 |
VS1T2037 |
VS1T2037 |
VS2T2043 |
VS2T2043 |
VS2T2050 |
VS2T2050 |
VS2T2020 |
VS2T2020 |
VS2T2021 |
VS2T2021 |
VS2T2028 |
VS2T2028 |
VS2T2029 |
VS2T2029 |
VS2T2036 |
VS2T2036 |
VS2T2037 |
VS2T2037 |
VS3T2043 |
VS3T2043 |
VS3T2050 |
VS3T2050 |
VS3T2020 |
VS3T2020 |
VS3T2021 |
VS3T2021 |
VS3T2028 |
VS3T2028 |
VS3T2029 |
VS3T2029 |
VS3T2036 |
VS3T2036 |
VS3T2037 |
VS3T2037 |
VS4T2043 |
VS4T2043 |
VS4T2050 |
VS4T2050 |
VS4T2020 |
VS4T2020 |
VS4T2021 |
VS4T2021 |
VS4T2028 |
VS4T2028 |
VS4T2029 |
VS4T2029 |
VS4T2036 |
VS4T2036 |
VS4T2037 |
VS4T2037 |
VS5T2043 |
VS5T2043 |
VS5T2050 |
VS5T2050 |
VS5T2020 |
VS5T2020 |
VS5T2021 |
VS5T2021 |
VS5T2028 |
VS5T2028 |
VS5T2029 |
VS5T2029 |
VS5T2036 |
VS5T2036 |
VS5T2037 |
VS5T2037 |
VS6T2043 |
VS6T2043 |
VS6T2050 |
VS6T2050 |
VS6T2020 |
VS6T2020 |
VS6T2021 |
VS6T2021 |
VS6T2028 |
VS6T2028 |
VS6T2029 |
VS6T2029 |
VS6T2036 |
VS6T2036 |
VS6T2037 |
VS6T2037 |
VS7T2043 |
VS7T2043 |
VS7T2050 |
VS7T2050 |
VS7T2020 |
VS7T2020 |
VS7T2021 |
VS7T2021 |
VS7T2028 |
VS7T2028 |
VS7T2029 |
VS7T2029 |
VS7T2036 |
VS7T2036 |
VS7T2037 |
VS7T2037 |
VS8T2043 |
VS8T2043 |
VS8T2050 |
VS8T2050 |
VS8T2020 |
VS8T2020 |
VS8T2021 |
VS8T2021 |
VS8T2028 |
VS8T2028 |
VS8T2029 |
VS8T2029 |
VS8T2036 |
VS8T2036 |
VS8T2037 |
VS8T2037 |
VS9T2043 |
VS9T2043 |
VS9T2050 |
VS9T2050 |
VS9T2020 |
VS9T2020 |
VS9T2021 |
VS9T2021 |
VS9T2028 |
VS9T2028 |
VS9T2029 |
VS9T2029 |
VS9T2036 |
VS9T2036 |
VS9T2037 |
VS9T2037 |
VSAT2043 |
VSAT2043 |
VSAT2050 |
VSAT2050 |
VSAT2020 |
VSAT2020 |
VSAT2021 |
VSAT2021 |
VSAT2028 |
VSAT2028 |
VSAT2029 |
VSAT2029 |
VSAT2036 |
VSAT2036 |
VSAT2037 |
VSAT2037 |
VSBT2043 |
VSBT2043 |
VSBT2050 |
VSBT2050 |
VSBT2020 |
VSBT2020 |
VSBT2021 |
VSBT2021 |
VSBT2028 |
VSBT2028 |
VSBT2029 |
VSBT2029 |
VSBT2036 |
VSBT2036 |
VSBT2037 |
VSBT2037 |
VSCT2043 |
VSCT2043 |
VSCT2050 |
VSCT2050 |
VSCT2020 |
VSCT2020 |
VSCT2021 |
VSCT2021 |
VSCT2028 |
VSCT2028 |
VSCT2029 |
VSCT2029 |
VSCT2036 |
VSCT2036 |
VSCT2037 |
VSCT2037 |
POS_X |
OEM_x |
POS_Y |
OEM_y |
POS_Z |
OEM_z |
VEL_X |
OEM_vx |
VEL_Y |
OEM_vy |
VEL_Z |
OEM_vz |
CALBLKID |
OSS_obsType |
CALBLKVR |
OSS_variant |
N_CCD |
Number of CCD |
BIASSEC |
Offset area relative to readout, 1-based |
CRPIXCNT |
fpa_cr_pixelcount |
CRPIXFRA |
ratioCosmics |
CRPIXDEN |
cosmicRayPixelDensity : density of cosmic by unity of time |
CRPIXOOL |
True if RatioCosmics > treshold_OOLCosmicRay, False otherwise |
MAGZEROP |
zero-point |
PHOT_ERR |
self calibration photometric error |
PHOTIRMS |
self calibration mag dispersion RMS |
APERCORR |
self calibration aperture correction MAG_50pix - MAG_13pix |
APCORRMS |
self calibration aperture correction rms error |
Extension Header For CalibratedFrame |
|
|---|---|
XTENSION |
IMAGE extension |
BITPIX |
Number of bits per data pixel |
NAXIS |
Number of data axes |
NAXIS1 |
Length of data axis 1 |
NAXIS2 |
Length of data axis 2 |
PCOUNT |
Required keyword; must = 0 |
GCOUNT |
Required keyword; must = 1 |
DETID |
CCD-ID field from science TM (0-35) |
CCDID |
e.g. Detector ID, e.g. ‘0-0’, ‘1-1’ … ‘6-6’ |
ROEID |
ROE ID: 1-12 |
ROECTV |
ROE configuration table version |
BUNIT |
Pixel data unit, e.g. ‘electrons’ or ‘adu’ |
PRESCANX |
number of serial prescan pixels, e.g 51 |
OVRSCANX |
number of serial overscan pixels, e.g 29 |
OVRSCANY |
number of parallel overscan pixels, e.g 20 |
EXPDUR1 |
Nominal/Short, nominal with CI, Flat: duration |
EXPDUR2 |
Nominal/Short, nominal with CI, Flat: duration |
CRVAL1 |
Right ascension at ref pixel |
CRVAL2 |
Declination at ref pixel |
CRPIX1 |
Reference pixel x coordinate |
CRPIX2 |
Reference pixel y coordinate |
CD1_1 |
Translation matrix element |
CD1_2 |
Translation matrix element |
CD2_1 |
Translation matrix element |
CD2_2 |
Translation matrix element |
CTYPE1 |
Coordinamte 1 type |
CTYPE2 |
Coordinamte 2 type |
CUNIT1 |
Physical units of CRVAL1 |
CUNIT2 |
Physical units of CRVAL2 |
WCSAXES |
Number of coordinate axes |
CMPRTSCI |
Quadrant image compression ratio obtained on-board |
GAIN |
Maximum equivalent gain (e-/ADU) |
RDNOISE |
Read out Noise |
RA |
Commanded FPA pointing right ascension (deg) |
DEC |
Commanded FPA pointing declination (deg) |
PA |
Commanded FPA pointing position angle (deg) |
EXPTIME |
Commanded integration time in seconds, e.g. 565.0 |
SATLEVEL |
Pixel saturation level in e- |
SATURATE |
Pixel saturation level in ADU |
BIAS_TOP |
bias_cfg[“main_bias_line_end”] |
BIAS_BTM |
bias_cfg[“main_bias_line_start”] |
BIAS_LFT |
bias_cfg[“main_bias_col_start”] |
BIAS_RGT |
bias_cfg[“main_bias_col_end”] |
GAINCORR |
ref_gain/cal_gain |
BIASLVLE |
Bias level quadrant E |
BIASRONE |
Bias readout noise quadrant E |
BIASLVLF |
Bias level quadrant F |
BIASRONF |
Bias readout noise quadrant F |
BIASLVLG |
Bias level quadrant G |
BIASRONG |
Bias readout noise quadrant G |
BIASLVLH |
Bias level quadrant H |
BIASRONH |
Bias readout noise quadrant H |
FLTCORR |
True if the LargeScaleFlat corection was applied |
STDCRMS |
|
NUMBRMS |
|
AVGRESID |
VIS mean residual |
PV1_1 |
|
PV2_1 |
|
PV1_2 |
|
PV2_2 |
|
PV1_4 |
|
PV2_4 |
|
PV1_5 |
|
PV2_5 |
|
PV1_6 |
|
PV2_6 |
|
PV1_7 |
|
PV2_7 |
|
PV1_8 |
|
PV2_8 |
|
PV1_9 |
|
PV2_9 |
|
PV1_10 |
|
PV2_10 |
|
RA_COMM |
commanding pointing RA |
DEC_COMM |
commanding pointing DEC |
PA_COMM |
commanding position angle |
REFCAT |
reference star catalogue file |
FPAMODEL |
fpa model file |
MAGZEROP |
zero-point |
PHOT_ERR |
self calibration photometric error |
PHOTIRMS |
self calibration mag dispersion RMS |
APERCORR |
self calibration aperture correction MAG_50pix - MAG_13pix |
APCORRMS |
self calibration aperture correction rms error |
FLXSCALE |
Flux scaling for stacked exposure |
EXTNAME |
Format: CCD row-CCD column.quadrant id, e.g. 2.3-H |
Weight Map¶
When combining multiple dithered exposures from separate detectors with gaps it becomes important to assign a “confidence” to each detector pixel so that overlapping exposures can be combined and the noise level at each pixel can be correctly estimated. This is the product to use as a weightmap when combining individual exposures into stacks.
Internally, the VIS processing function uses relative “weight maps”, which are defined as \(w_j \propto \frac{1}{{\sigma_{j}}^{2}}\), where \(\sigma_j\) is the standard deviation (or “RMS”) of the \(j_{th}\) pixel.
At the end it corespond to the small scale flat (PRNU) with all pixels with invalid flag set to 0. This corespond only to the background, not the object.
Data Header¶
Primary Header for Weight Map |
|
|---|---|
SIMPLE |
conforms to FITS standard |
BITPIX |
array data type |
NAXIS |
number of array dimensions |
EXTEND |
|
Extension Header for Weight Map |
|
|---|---|
XTENSION |
Image extension |
BITPIX |
array data type |
NAXIS |
number of array dimensions |
NAXIS1 |
|
NAXIS2 |
|
PCOUNT |
Number of parameters |
GCOUNT |
Number of groups |
EXTNAME |
|
CCDID |
|
WCSAXES |
|
CTYPE1 |
Coordinamte 1 type |
CTYPE2 |
Coordinamte 2 type |
CUNIT1 |
Physical units of CRVAL1 |
CUNIT2 |
Physical units of CRVAL2 |
CRPIX1 |
|
CRPIX2 |
|
CD1_1 |
|
CD1_2 |
|
CD2_1 |
|
CD2_2 |
|
CRVAL1 |
|
CRVAL2 |
|
STDCRMS |
|
NUMBRMS |
|
AVGRESID |
|
PV1_1 |
|
PV2_1 |
|
PV1_2 |
|
PV2_2 |
|
PV1_4 |
|
PV2_4 |
|
PV1_5 |
|
PV2_5 |
|
PV1_6 |
|
PV2_6 |
|
PV1_7 |
|
PV2_7 |
|
PV1_8 |
|
PV2_8 |
|
PV1_9 |
|
PV2_9 |
|
PV1_10 |
|
PV2_10 |
|
RA_COMM |
commanding pointing RA |
DEC_COMM |
commanding pointing DEC |
PA_COMM |
commanding position angle |
RA |
reconstructed pointing RA |
DEC |
reconstructed pointing DEC |
PA |
reconstructed position angle |
REFCAT |
reference star catalogue fi |
FPAMODEL |
fpa model file |
Background Map¶
The background map delivered corespond to the background map from SExtractor. This FITS file as the same pixel size as the individual images.
Data Header¶
Primary Header for Background Map |
|
|---|---|
SIMPLE |
This is a FITS file |
BITPIX |
|
NAXIS |
|
EXTEND |
This file may contain FITS extensions |
NEXTEND |
Number of extensions |
MAGZEROP |
zero-point |
PHOT_ERR |
self calibration photometric error |
PHOTIRMS |
self calibration mag dispersion RMS |
APERCORR |
self calibration aperture correction MAG_50pix - MAG_13pix |
APCORRMS |
self calibration aperture correction rms error |
EXPTIME |
Commanded integration time in seconds, e.g. 565.0 |
Extension Header for Background Map |
|
|---|---|
XTENSION |
Image extension |
BITPIX |
array data type |
NAXIS |
number of array dimensions |
NAXIS1 |
|
NAXIS2 |
|
PCOUNT |
Number of parameters |
GCOUNT |
Number of groups |
EXTNAME |
|
DETID |
CCD-ID field from science TM (0-35) |
CCDID |
Detector ID, e.g. ‘0-0’, ‘1-1’ … ‘6-6’ |
ROEID |
ROE ID: 1-12 |
ROECTV |
ROE configuration table version |
BUNIT |
Pixel data unit, e.g. ‘electrons’ or ‘adu’ |
PRESCANX |
Number of serial prescan pixels, e.g 51 |
OVRSCANX |
Number of serial overscan pixels, e.g 29 |
OVRSCANY |
Number of parallel overscan pixels, e.g 20 |
EXPDUR1 |
Nominal/Short, nominal with CI, Flat: duration |
EXPDUR2 |
Nominal/Short, nominal with CI, Flat: duration |
CRVAL1 |
|
CRVAL2 |
|
CRPIX1 |
|
CRPIX2 |
|
CD1_1 |
|
CD1_2 |
|
CD2_1 |
|
CD2_2 |
|
CTYPE1 |
Coordinamte 1 type |
CTYPE2 |
Coordinamte 2 type |
CUNIT1 |
Physical units of CRVAL1 |
CUNIT2 |
Physical units of CRVAL2 |
WCSAXES |
|
CMPRTSCI |
Quadrant image compression ratio obtained on-board |
GAIN |
|
RDNOISE |
|
RA |
reconstructed pointing RA |
DEC |
reconstructed pointing DEC |
PA |
reconstructed position angle |
EXPTIME |
Commanded integration time in seconds, e.g. 565.0 |
SATLEVEL |
pixel saturation level in e- |
SATURATE |
pixel saturation level in ADU |
BIAS_TOP |
|
BIAS_BTM |
|
BIAS_LFT |
|
BIAS_RGT |
|
GAINCORR |
ref_gain/cal_gain |
BIASLVLE |
|
BIASRONE |
|
BIASLVLF |
|
BIASRONF |
|
BIASLVLG |
|
BIASRONG |
|
BIASLVLH |
|
BIASRONH |
|
FLTCORR |
|
STDCRMS |
|
NUMBRMS |
|
AVGRESID |
|
PV1_1 |
|
PV2_1 |
|
PV1_2 |
|
PV2_2 |
|
PV1_4 |
|
PV2_4 |
|
PV1_5 |
|
PV2_5 |
|
PV1_6 |
|
PV2_6 |
|
PV1_7 |
|
PV2_7 |
|
PV1_8 |
|
PV2_8 |
|
PV1_9 |
|
PV2_9 |
|
PV1_10 |
|
PV2_10 |
|
RA_COMM |
Commanding pointing RA |
DEC_COMM |
Commanding pointing DEC |
PA_COMM |
Commanding position angle |
REFCAT |
reference star catalogue file |
FPAMODEL |
|
MAGZEROP |
zero-point |
PHOT_ERR |
self calibration photometric error |
PHOTIRMS |
self calibration mag dispersion RMS |
APERCORR |
self calibration aperture correction MAG_50pix - MAG_13pix |
APCORRMS |
self calibration aperture correction rms error |
FLXSCALE |
Flux scaling for stacked exposure |
CCD_RA |
CCD center right ascension |
CCD_DEC |
CCD center declination |
BKG_MED |
CCD bkg median value in MJY/sr |
BKG_MAD |
CCD bkg median absolute deviation in MJy/sr |