ESA Datalabs is an innovative e–science platform that provides users the capability to bring their code to ESA’s infrastructure, offering direct access to ESA’s extensive archives. A unique feature of the platform is the creation of specialized software applications called datalabs, which function as comprehensive computational environments. The underlying architecture of ESA Datalabs supports a wide range of collaborative research activities by integrating access to big data, Jupyter notebook technologies, and domain-specific software. The Euclid Datalab, which is based on JupyterLab, is designed to facilitate direct access to Euclid Q1 data and Euclid Early Release Observations (EROs) without necessitating data downloads. The platform encourages users to perform their data analysis directly on the system; although small user-derived data products can be downloaded. Users have access to personal storage space of 100 GB, my workspace, and shared collaborative spaces named team workspaces. Team workspaces are ideal for handling larger data volumes and can be set up through a request to the Service Desk. The Euclid Datalab includes a Jupyter kernel called euclid-tools (a conda environment) that comes preloaded with various scientific and astronomy-specific packages, such as Astroquery, Astropy, Fitsio, Jdaviz, Matplotlib, Numpy, Pandas, Photutils, Pyarrow, PyESASky, Seaborn, SEP, Scipy, Astromatic-scamp, Astromatic-swarp, Astromatic-source-extractor, Astrometry.net, among others. Users can install additional packages via the terminal or Jupyter Notebook, and use conda environments, but these packages will need to be reinstalled if a new datalab instance is launched. Regarding resources, each user can run up to two datalabs simultaneously. For more sessions, existing instances must be first deleted. The Euclid datalab provides 32 GB RAM and 2 CPU cores per instance, with personal storage space of 100 GB. Euclid data, including Q1 (35 TB) and ERO data (600 GB), is available on the platform as data volumes, which are automatically mounted. This setup allows users to access data directly without downloading it, making it more efficient for analysis. Example tutorial Jupyter notebooks are included in the data volume to aid users in their analysis. The example notebooks cover a range of topics, including accessing Euclid data using the Euclid Astroquery package, querying Euclid data with ADQL, generating single image cutouts and in-bulk, performing source extraction, and visualizing images and spectra staticly or interactively. Additionally, specific notebooks for Euclid ERO data are provided, offering guidance on accessing ERO data and performing image colorisation tasks. Users are encouraged to use these notebooks as introductory materials and starting points to develop their own notebooks, tailored to individual use cases. The notebooks can be shared with collaborators on team workspaces. For any issues, queries or suggestions please reach out via Euclid Helpdesk. Access to ESA Datalabs requires an ESA Cosmos account. If you do not have a Cosmos account, you can self-register here. To request an ESA Datalabs account:
Visit the ESA Datalabs Self-Registration.
Sign in with your ESA Cosmos credentials.
Enter the invitation code: EUCLIDQ1.
Submit your request (approval may take a couple of working days).
Since ESA Datalabs is in Public Moderated Beta, registrations are not automatic and require prior approval. The invitation code is mandatory for approval. Important note: ESA Datalabs is still in development, and this access is experimental. Users may encounter downtime and system instability, which could impact user experience. Primary access to Q1 data remains via the Euclid Science Archive. ESA Datalabs should not be used to download Euclid Q1 data products. Feedback from users through the Euclid Helpdesk is encouraged to improve future releases. For any scientific publication resulting from the use of ESA Datalabs resources, users should include a citation to ESA Datalabs (Navarro et al. 2024) and/or DOI: 11 1 Navarro, V. et al. (2024). ESA Datalabs: Digital Innovation in Space Science. In: Cortesi, A. (eds) Space Data Management. Studies in Big Data, vol 141. (https://doi.org/10.1007/978-981-97-0041-7_1). Additionally, the acknowledgment section of the paper should contain the following reference: ”This research makes use of ESA Datalabs (datalabs.esa.int), an initiative by ESA’s Data Science and Archives Division in the Science and Operations Department, Directorate of Science.”