NIR Calibrated Frame

$SetSchemaPath nir/euc-nir-CalibratedFrame.xsd

Data product name

$PrintDataProductName

Data product custodian

$PrintDataProductCustodian

Name of the Schema File

$PrintSchemaFilename

Last Edited for DPDD Version

1.1

Processing Elements Creating / Updating / Using the Product

Creators:

• The full NIR Pipeline

Consumers:

• NIR_Resampling

Processing Function using the products

• PF-NIR, PF-MER, PF-SIR

Proposed for inclusion in EAS/SAS

This product is proposed for inclusion in the SAS: yes

This is one of the main PF-NIR products and represents a single NISP Raw Frame Product image fully calibrated.

Data Product Elements

$PrintDataProductElements

Detailed Description of the Data Product

The DpdNirCalibratedFrame is the OU-NIR Data Product produced by the execution of PF-NIR scientific pipeline on a NISP Raw Frame Product image. Pipeline tasks correct and flag the image for instrumental signatures and provides a fully astrometric and photometric calibrated image.

This Data Product is used by the Processing Function MER as input for its pipeline.

Fig. 15 : Graphical representaton of Data section structure for DpdNirCalibratedFrame

The DpdNirCalibratedFrame Data section, reported on image Fig. 15, is structured in classes with increasing specificity on the data product.

• The most generic imgBaseFrame collects all metadata relative to the image structure itself as the Imgtype and data size.

• Metadata contained in img:singleExposure report the information on Filters, Spatial Footprint and the Euclid bit Mask adopted. DetectorList hosts the characterization of each detector, as the detector Id, the WCS parameters and the photometrci zero point.

• Metadata in the nir:nirCalibratedFrame structure contain:

  • the Observation Sequence, that are numbers needed to identify the specific pointing, the date, the ReadoutMode adopted and the TargetPointing

  • the name of the 4 files in the Storage metadata (see below)

  • a section dedicated to the QualityParams, a set of statistical values and validation parameters useful to evaluate the data quality and identify anomalies

The DpdNirCalibratedFrame Data Product references to 4 files:

1. Scientific Image in Data.DataStorage: a multi-extension FITS (MEF) file, with FitsFormat id = “nir.calibratedScienceFrame”, containing three extensions per detector representing calibrated science image, RMS, and Data Quality flags. Hence the final MEF file has 49 layers: a primary header + 3 extensions for each of the 16 detectors.

2. Background model in Data.BackgroundStorage: a MEF file with the same structure as the Scientific Image, containing the estimated background.

3. a PSF Model in Data.PsfStorage.PsfModelFile: a Point Spread Function (PSF) model as provided by PSFEx software (.psf).

4. a PSF Image in Data.PsfStorage.PsfFITSFile: a single extension FITS file containing the PSF image associated to the science image. It can be extended to include wavelength and focal plane dependency.

Each file is provided toghether with a dedicated Quality Parameter section; therefore the Data.QualityParams is structured in 3 sections: ScienceQualityParams, BackgroundQualityParams and PsfQualityParams for science, background and PSF respectively.

Scientific image FITS file

The main FITS file associated to this Data Product, linked on Data.DataStorage, contains the scientific image derived from the application of PF-NIR pipeline NIR Pipeline on the NISP Raw Frame Product data product.

The FITS file structure, therefore, is an evolution of the NISP Raw Frame Product one. It has a primary header with main informations on poitings and 3 layer for each detector, 49 layers in total. The layer name, in EXTNAME keyword, follow the convention DETxy.SCI, DETxy.RMS, and DETxy.DQ, for Science, RMS, and Data Quality layers respectively. xy are the detector coordinated on the NISP Detector System Assembly as illustrated on :numref:`NISP_detector_position_ids`image.

Layer name	Data Format	Units	Content
PRIMARY			Main headers
DETxy.SCI	float32	electron	Scientific data
DETxy.RMS	float32	electron	RMS data
DETxy.DQ	int32		Data Quality mask

Fig. 16 : Graphical representation of NIR calibrated frames 3-layer per detector structure

Note that background is not subtracted from science images, and pixel values are still in native units (electrons). Relative and absolute calibration factors for each detector, for conversion to microJy, are included in the metadata, as well as zero-points for conversion to AB magnitudes.

Reference pixels, the border of 4 not exposed pixels around the image, are removed from all detector, which size is therefore 2040x2040 pixels.

Primary layer

The PRIMARY header keywords report main observational pointing and instrumental information and other parameters relative to the NIR pipeline execution.

$PrintPrimaryHeader fit/euc-nir.xml nir.calibratedScienceFrame

Scientific layer

The DETxy.SCI layer contains the input image corrected from instrumental effects and converted in electrons. The astrometrical and photometrical correction factors are reported on detector header.

$PrintExtensionHeader fit/euc-nir.xml nir.calibratedScienceFrame DETn.SCI

Standard Deviation layer

The pixel value stored into the DETxy.RMS data layer is derived from a variance layer intenally used by NIR Pipeline tasks. It is initially instantiated by the NIR Init task application according the formula 12 of Kubik (2016) paper.

The final variance layer value is the result of the calibration file contributions added during the application of NIR Pipeline tasks.

Data Quality layer

The DETxy.DQ layer is a bit maks containing flags associated to each pixel during the pipeline execution.

The first bit INVALID identifies pixels that are considered not to be used. It is activated by any pipeline task that identify a critical feature (i.e. bad pixels, cosmic hits and saturaded pixels).

The single bitmask flag description is available in dedicated wiki and shown in Fig. 17

Fig. 17 : NIR Data Quality Bit value, flag name and description.

Background FITS file

The Background FITS file contains the background value estimated from images by the NIR Background estimation task.

The FITS file structure mirrors the scientific file one with 49 layers: a primary header with main information and 3 layers (SCI, RMS and DQ) for each detector.

PSF model and image files

The NIR PSF estimation task estimates a PSF model from scientific images thanks a PSFEx run.

The software produces a .psf file containing model parameters and a .fits with an image of the PSF model. This last product is used by PF-MER.

Data Quality Parametes

The Data.QualityParams section provide a set of Quality Parameters derived from each of the files.

The Data.QualityParams.ScienceQualityParams contains a set of statistical values (min, max, mean, median std and masked pixel fraction) for the whole image and some Quality Parameters correlated to the astrometric and photometric calibration. For each detector are indeed provided the same set of statistics and the numbers of pixel flagged for each bit mask value.

The Data.QualityParams.BackgroundQualityParams contains the set of statistical values for each detector. Despite the background image pixel value is in [electron], the background detector statistical values are converted in [Myr/sr].