Science Exploitation in a Big Data Archive: the Euclid Scientific Archive System

Euclid is an ESA mission and a milestone in the understanding of the geometry of the Universe. The Euclid Archive System (EAS) is a joint development between ESA and the Euclid Consortium and is led by the Science Data Centres (SDC) of the Netherlands and the ESDC (ESAC Science Data Centre). Big-data technologies, driven by data nature and volume, are transforming the way of doing scientific research towards collaborative platforms whose first goal is to enable access and process large data sets in ways that could not be done downloading the data. Some examples of the main technologies explored as part of the Euclid scientific archive are: JupyterLab, Apache Spark, GreenPlum and PostgresXL.