NISP Raw Frame Product

Data Product Name

DpdNispRawFrame

Data Product Custodian

LE1

Name of the Schema File

euc-le1-NispRawFrame.xsd

Last Edited for DPDD Version

1.1

Processing Elements Creating / Updating / Using the Product

Creators:

• LE1 NISP Processor

Consumers:

• NIR, SIR, SOC AUX Processor, SOC QLA, IODA

Processing Function using the products

LE1

Data Product Elements

Header:

object of type sys:genericHeader

Data:

object of type le1nisp:nispRawFrame

QualityFlags:

object of type dqc:sqfPlaceHolder

Parameters:

object of type ppr:genericKeyValueParameters

Detailed Description of the Data Product

A NISP raw exposure provides calibration, engineering and scientific exposures acquired by the NISP instrument, together with the associated metadata, for each supported exposure configuration mode.

The following table summarizes the NISP instrument exposure configurations and the corresponding raw data sections available in the files produced by the on-board Data Processing Unit (DPU):

Fig. 3 : NISP exposure configurations and corresponding data sections

Depending on the exposure configuration, three types of FITS files can be produced and referenced by a single DpdNispRawFrame data product:

1. NISP frame: it is a sngle multi-extension FITS file, with FitsFormat id = “le1.nispRawImage”, containing two extensions per detector; one extension is an image containing the detector scientific data, while a second extension provides the detector Chi2 data. Hence the file can contain up to 32 extensions. The Chi2 data content depends on the instrument channel: for photometric exposures, it provides a quality value {0,1} computed on-board, while for the spectroscopic channel it includes the full Chi2 value (8-bit integer).

2. HK raw data: it is a single multi-extension FITS file, with FitsFormad id = “le1.nispHkRaw”, which includes housekeeping information extrated from the raw DPU file, including digital telemetry, raw lines for each detector, analog telemetry, error buffer and history data (see the NISP Instrument Flight User Manual Document, section 14.1).

3. Engineering exposures: it is a list of FITS files, with FitsFormat id = “le1.nispEngExposure”. Each file includes one or more image HDUs. The meaning and structure of these images depend on the specific engineering mode (raw or debug) configured for the exposure acquisition.

NISP Frame FITS format

Each NISP exposure is provided as one multi-extension FITS (MEF) file. In particular, the file contains two extensions per detector: one extension for the detector scientific data, and one extension for the detector Chi2 data (quality factor). Depending on the number of detectors acquired, the file can contain up to 32 extensions. Tipically, the engineering model of the instrument only has few detectors connected, while the flight model provides all 16 detectors.

The detectors numbering and location on the NISP Detector System Assembly is shown in Fig. 4 below

Fig. 4 : NISP Detector System Assembly: detectors position and ID

Within each Sensor Chip Array (SCA), the pixel (0,0) is located on the upper left corner when seeing the front side of the SCA with their wire bounding located on top. Consequently, pixel (0,0) is located:

• on the upper left corner in the detectors:

  • #11, #12, #13, #14

  • #21, #22, #23, #24

• on the lower right corner in detectors:

  • #31, #32, #33, #34

  • #41, #42, #43, #44

This is shown in Fig. 5

Fig. 5 : Pixel layout of the different SCAs

Each SCA contains both reference pixels and science pixels. Reference pixels are located all around the detector. The science pixels are located in the central window:

(xcorner, ycorner, xsize, ysize) = (3, 3, 2040, 2040)

Data encoding

For each detector:

• pixels in the scientific extension are in units of ADU, 16-bit unsigned integer.

• pixels in the Chi2 extension only provides a single bit of information for the photometric observations, and an integer value for the spectroscopic observations. In both cases the values are stored as an 8-bit unsigned integer.

Primary header

HDU PRIMARY: header
	Name	Description	Value
Keywords	FITS_DEF	FITS definition name	string = le1.nispRawImage
	FITS_VER	FITS definition version	string = 1.0
	TELESCOP		string = Euclid
	INSTRUME	or NISPsim for simulations	string = NISP
	VERSION	Data Release version	string
	DATE	UT date when this file was created	string
	ORIGIN	FITS file originator	string
	OBASW	On board application software version (NISP DPU SW version)	string
	SOFTVERS	Version of the processing function	string
	AUX_VERS	Version of the AUX Processor	string
	OBT_STA1	Integer, number of seconds of OBT (START_TIME)	integer
	OBT_STA2	Integer, number of microseconds of OBT (START_TIME)	integer
	DATE-OBS	UTC Date of observation (start), e.g. ‘2014-03-15T09:30:09.313’	string
	MJD-OBS	MJD start time, e.g. 56731.39594113	double
	DATE_AUX	ISO-format timestamp when AUX Processor has added additional metadata	string
	IMG_CAT	DP category (SCIENCE, CALIB, …)	string
	IMG_T1	DP type (OBJ, SRD, DARK, …)	string
	IMG_T2	DP type (LAMP, SKY, …)	string
	OBSTYPE	DP technique (IMAGE, SPECTROIMAGE)	string
	OBSMODE	Observation Mode (WIDE, DEEP, CALIBRATION)	string
	READMODE	Readout Mode Method: (Multiaccum, UpTheRamp, FowlerSamples)	string
	NR	MACC Number of reads	integer
	NG	MACC Number of Groups	integer
	ND	MACC Number of drops	integer
	FRTIME	Exposure time in seconds for single readout sample. Default value =1.45408 sec	double
	LINETIME	Time in seconds to read a single line of 2048 pixels. Default value = 710E-6	double
	EXPTIME	Effective integration time in seconds, e.g. 60.0005	double
	ELAPTIME	Elapsed time of the observation in seconds	double
	RA	Commanded FPA pointing right ascension (deg)	double
	DEC	Commanded FPA pointing declination (deg)	double
	PA	Position angle (deg). The rotation of the FPA about the line of sight, east of north	double
	EQUINOX	Standard FK5 (years), e.g. 2000.	double
	RADECSYS	Coordinate reference frame, e.g. ‘FK5’	string
	ELONG	Ecliptic longitude of the commanded attitude	double
	ELAT	Ecliptic latitude of the commanded attitude	double
	POS	Position Angle of the commanded attitude	double
	SAA	Solar Aspect Angle of the commanded attitude	double
	ALPHA	Solar Alpha angle, in degrees, at the start of the exposure (commanded)	double
	BETA	Solar Alpha angle, in degrees, at the start of the exposure (commanded)	double
	PLAN_ID	[0:1] SOC Planning ID	integer
	PATCH_ID	Patch ID	integer
	OBS_ID	Observation ID, e.g. 1	integer
	DITHOBS	Number sequence of dithering, e.g. 1	integer
	PTGID	Integer number defining exposure inside observation	integer
	CALBLKID	Calibration Block ID	string
	EXPNUM	Internal integer counter for single images	integer
	TOTEXP	Total number of Exposures in specific pointing	integer
	FWA_POS	FWA position (J,H,Y,OPEN,CLOSED, HOMECHECK)	string
	FWA_ANG	Commanded Filter Wheel Angle in the range [0,360[	integer
	FWA_REF	Filter Wheel Angle Reference position to compute angle	string
	GWA_POS	GWA position (RGS270,RGS000,RGS180,BGS000,OPEN, HOMECHECK)	string
	GWA_ANG	Commanded Grism Angle in the range [0,360[	integer
	GWA_REF	Grism Wheel Angle Reference position to compute angle	string
	GWA_TILT	Commanded Grism Tilt Angle	integer
	KEYS_CNF	Keys configuration. Derived from EXP_CNF. Enumerative	string
	INST_CNF	Instrument configuration. Derived from EXP_CNF. Enumerative	string
	CU_STATE	NI-CU state: ON=1,OFF=0	logical
	LED_ID	LED ID (A,B,C,D,E,U), from HK TM	string
	LED_INT	LED Intensity [mA]	double
	LED_PWM	LED PWM Duty Cycle (pct.)	double
	FLUX_ID	LED Flux ID	integer
	ACQ_CNT	Acquisition Counter from raw header	integer
	EXP_CNF	[5]: Exposure configuration mode	integer
	T_RESETS	[8]: Current Ramp total Reset Frames	integer
T_DROPL1	[10]: Current Ramp total Post Reset Drop Lines	integer
T_DROPL2	[12:13]: PostRead DropLines	integer
S_OFFSET	[55]: Signal Offset	integer
S_FACTOR	[56]: Signal Scaling Factor	integer
RPIXPRC1	DPU1 Reference Pixel proc mode	string
RPIXPRC2	DPU2 Reference Pixel proc mode	string
NIST0385	LED Status (ON/OFF, 1 bit per LED)	integer
NIST0642	CU ASW - DPU1 ASW Processing Mode, from HK TM	integer
NIST4738	CU ASW - DPU2 ASW Processing Mode, from HK TM	integer
NIST0485	LED A voltage [V], from HK TM	double
NIST0486	LED B voltage [V], from HK TM	double
NIST0487	LED C voltage [V], from HK TM	double
NIST0488	LED D voltage [V], from HK TM	double
NIST0489	LED E voltage [V], from HK TM	double
WCCT3290	Nominal ICU voltage from S/C HK	double
WCCT3291	Nominal ICU current from S/C HK	double
WCCT3316	Redundant ICU voltage from S/C HK	double
WCCT3317	Redundant ICU current from S/C HK	double

Extensions

For each NISP detector, two HDUs are provided: a science “layer”, containing the pixel data, and a data quality layer that, for the photometric data, will just contain 1 bit of information per pixel, reporting the cosmic-ray hits detected on-board.

Science layer:

HDU SCI.n: header
	Name	Description	Value
Keywords	EXTNAME	e.g. ‘DET13.SCI’, meaning Detector ‘11’ Science data	string
	DET_ID	Detector ID, e.g. ‘13’	string
	SCEINDEX	SCE ID from raw header	integer
	SCA_ID	SCA full name given by NASA	string
	CRVAL1	Right ascension at ref pixel	double
	CRVAL2	Declination at ref pixel	double
	CRPIX1	Reference pixel x coordinate	double
	CRPIX2	Reference pixel y coordinate	double
	CTYPE1	Coordinamte 1 type	string
	CTYPE2	Coordinamte 2 type	string
	CUNIT1	Physical units of CRVAL1	string
	CUNIT2	Physical units of CRVAL2	string
	CD1_1	Translation matrix element	double
	CD1_2	Translation matrix element	double
	CD2_1	Translation matrix element	double
	CD2_2	Translation matrix element	double
	BUNIT	Pixel data unit, e.g. ‘ADU’	string
	RON_DET	[40]: Detector ReadOutNoise in Proc Param Table	integer
	GAIN_DET	[41]: Detector GainFactor in Proc Param Table	integer
	DTEXPNUM	[50] N. of ktc_exp cycles or sce_exp frames	integer
	DPU_ID	[51a] DPU ID	integer
	MASTER	[51b] Master DPU	integer
	CMPRTSCI	Detector image compression ratio obtained on-board	double

Quality layer:

HDU CHI2.n: header
	Name	Description	Value
Keywords	EXTNAME	e.g. ‘DET11.DQ or DET11.CHI2’, meaning Detector ‘11’, with chi2 data (boolean value or full slope)	string
	DET_ID	Detector ID, e.g. ‘13’	string
	CMPRTX2	CHI2 compression ratio obtained on-board	double

NISP HK raw data FITS format

The HK (HouseKeeping) raw file provides auxiliary information for a given exposure. It is a multi-extension FITS file with a group of HDUs per detector. It provides data coming from several sections of the DPU file:

• digital telemetry parameters

• analog telemetry parameters

• raw lines

• history buffer (i.e. a set of samples for one selected analog parameter)

• error buffer

Primary header

The primary header contains digital telemetry parameters with values shared by all detectors:

HDU PRIMARY: header
	Name	Description	Value
Keywords	FITS_DEF	FITS definition name	string = le1.nispHkRaw
	FITS_VER	FITS definition version	string = 1.0
	TELESCOP		string = Euclid
	INSTRUME	or NISPsim for simulations	string = NISP
	VERSION	Data Release version	string
	DATE	UT date when this file was created	string
	ORIGIN	FITS file originator	string
	OBASW	On board application software version (NISP DPU SW version)	string
	SOFTVERS	Version of the processing function	string
	AUX_VERS	Version of the AUX Processor	string
	OBT_STA1	Integer, number of seconds of OBT (START_TIME)	integer
	OBT_STA2	Integer, number of microseconds of OBT (START_TIME)	integer
	PLAN_ID	[0:1]: SOC Planning ID	integer
	OBT1	[2]: ACQ Start Gros Seconds	integer
	OBT2	[3]: ACS Start Seconds	integer
	OBT3	[4]: ACQ Start subseconds	integer
	EXP_CNF	[5]: Exposure configuration mode	integer
	T_RESETS	[8]: Current Ramp total Reset Frames	integer
	T_READS	[9]: Current Ramp total Read Frames	integer
	T_DROPL1	[10]: Current Ramp total Post Reset Drop Lines	integer
	T_GROUPS	[11]: Acquisition Groups	integer
	T_DROPL2	[12:13]: PostRead DropLines	integer
	RESETS	[14]: Current Reset Frame counter	integer
	READS	[15]: Current Read Frame counter	integer
	DROPL1	[16]: Current Post Reset Drop Lines counter	integer
	CGROUPS	[17]: Current Groups Counter	integer
	DROPL2	[18:19]: CurRamp PostRead DropLines	integer
	MODESTAT	[23]: ModeStatus	integer
	RAMPSTAT	[24]: RampStatus Digital TLM [35:39] are reserved and not used yet	integer
	CMP_BLKS	[44]: Used Compr. Block Size	integer
	CMPFSBIT	[45]: Used Compr. Fund. Seq. Par1	integer
	CMPFSMAX	[46]: Used Compr. Fund. Seq. Par2	integer
	TMTAGDEL	[47]: Time Tag Delay Digital TLM [48:49] are reserved and not used yet	integer
	S_OFFSET	[55]: Signal Offset	integer
	S_FACTOR	[56]: Signal Scaling Factor	integer
	TLMHISTP	[63]: History Parameter index (from ASW)	integer
	NIST0385	LED Status (ON/OFF, 1 bit per LED)	integer
	NIST0642	CU ASW - DPU1 ASW Processing Mode, from HK TM	integer
	NIST4738	CU ASW - DPU2 ASW Processing Mode, from HK TM	integer
	NIST0485	LED A voltage [V], from HK TM	double
	NIST0486	LED B voltage [V], from HK TM	double
	NIST0487	LED C voltage [V], from HK TM	double
	NIST0488	LED D voltage [V], from HK TM	double
	NIST0489	LED E voltage [V], from HK TM	double
	WCCT3290	Nominal ICU voltage from S/C HK	double
	WCCT3291	Nominal ICU current from S/C HK	double
	WCCT3316	Redundant ICU voltage from S/C HK	double
	WCCT3317	Redundant ICU current from S/C HK	double

Extensions

For each detector, a group of 3 HDUs are provided. The first HDU, named DET<XX>.RAW is a binary table that includes:

• a header containing digital telemetry parameters specific for the detector and analog telemetry parameters;

• a table providing a full MACC ramp (excluding drop frames) for few selected lines of the detector. Each raw line is an entire detector row (i.e. 2048 pixels), and it is saved for each frame in the multi-accumulation readout. The corresponding FITS table contains a total of (Number of raw lines) * (Group Counter) * (Frame per Group) entries.

The second HDU of the group, named DET<XX>.ERR, contains the readout frames/lines errors (1024 bytes) retrieved from the DCU x the maximum number of groups in the MACC acquisition. It is stored as an image HDU of size (n. groups, 1024).

The third HDU, named DET<XX>.HIST, contains the complete history, recorded frame by frame (32 words x number of groups), of a selected parameter (whose ID is in parameter 63 of the Digital Telemetry list), as programmed in DCU_SRLO command.

Engineering exposure FITS format

The Engineering exposure FITS files stores the detector frames acquired with two engineering modes of the NISP Instrument:

• Engineering debug mode: this mode allows to download to ground, in addition to the standard frame transmission, all the un-compressed groups of the exposure for a subset (up to 2%) of pixels for all detectors;

• Engineering raw mode: this mode allows to store and transfer all un-processed group frames of one full detector plus the result of the processing (final signals and QF). Therefore, this mode can be applied to two detectors at a time (one detector per DPU), due to the bandwith limits.

The use of the Engineering exposurse FITS files depends on the engineering mode used:

• for the Engineering debug mode, the LE1 NISP Processor outputs a single Engineering exposure FITS file with a number of HDUs equal to the number of detectors. Each HDU, named DET<XX>.ENG, is an image HDU with size (number of selected pixels, number of groups);

• for the Engineering raw mode, the LE1 NISP Processor outputs one Engineering exposure FITS file per detector (up to 2 detectors in this mode). Each file has a number of Image HDUs equal to the number of groups in the MACC acquisition, named DET<XX>.GROUP<N>.ENG. Each image has the nomianl NISP detector size, i.e. (2048x2048) since it provides the unprocessed detector frame for each group of the MACC acquisition.

The primary header mostly replicates the header of the NISP Frame FITS format and analogously, the extension headers mostly replicate the detector headers.