Purpose and scope¶
Purpose¶
This is the Data Product Description Document. It aims at providing a comprehensive list of all the data products that are created and manipulated in the context of the Science Ground Segment data processing pipeline (or “SGS pipeline” in short), accompanying this list with information on the data products themselves.
In the present context a Data Product, or product, is a complex data structure that may contain different type of values, keyword-based information, and/or meta-data. While no strict definition of a product can really be made, these are defined by the developers of the processing function, and they are the basic objects that the EAS will handle, store and reference.
The DPDD should be viewed as an encyclopedia of Euclid products. There is one single entry per products, and the products are grouped by the PF that produces them. We envisioned two ways to use the DPDD:
As an encyclopdedia, where a user will find a product in the EAS and look for informations about it in the DPDD
As a manual where the user will browse through the products of a given PF to determine which best suits he needs for his research.
Historically, this document was first used as a support for the developpers of the SGS pipeline to understand the products coming from the other PF that they are using in their own PF. It is also now used as a reference documents about roducts for the general science users of Euclid. While developpers work on the DPS, science users are accessing the archive through the SAS. The SAS only contains a subset of the profusion of data products generated by the SGS. Moreover, these products are often stripped of a lot of their metadata, only keeping the core FITS tables or images. Nevertheless, the DPDD is supposed to be comprehensive, so all bits of a product should be documented here, especially if they are in the SAS. Also, if users belive some products not included in the SAS should be distributed, please let the SGS scientist know so their inclusion can be discussed with ESA in future releases of the SAS.
DPDD is a companion document to the SGS Common Data Model, a software component that defines the structure of the data products used by the SGS Pipeline.
Scope¶
This version of the DPDD is to accompgny the Q1 public data release. It lists the products served by SAS.
For each product, we provide a detailed description of the product’s content, to allow in-depth understanding of the product content, as well as information on where the product is created, and where it is going to be used. We present this in the form of data product “cards”, that standardize the organization of the information.
As will be evident in browsing this document, there is a large number of data products created and handled by the SGS pipeline. This is expected for such a deep processing system. Given that the development of the pipeline is distributed (i.e. the different Processing Functions are developed by independent groups of scientists and developers), there is an intrinsic heterogeneity in the system. The Euclid Consortium as it is built does not have the mean to fully control this heterogeneity (by providing for instance centralized development of given classes of data products), however we have put in place some measures to try an maintain it to an acceptable level.