Managing Earth Observation datasets as multidimensional arrays using SciDB and open standards

Earth Observation (EO) datasets commonly provide scenes for particular regions of the Earth's surface. Each scene consists of multiple raster satellite images representing the space at individual time stamps. A key challenge in managing EO datasets is providing fast access to 2-dimensional coverages periodically collected. A representation concept of 3-dimensional space-time Data Cubes and an approach for providing dynamic and on-demand delivery of 3-dimensional subsets, increases the interoperability and usability for EO datasets. As an extreme scenario of subsetting, also time series of single pixels can be retrieved for a detailed analysis or visualisation. In a prototypical development, we extract 2-dimensional gridded pixel data from individual satellite images of single time stamps and insert them into a 3-dimensional space-time array using SciDB, a multidimensional array database. On arrival of a new satellite image, we transform it's pixel data into a 2-dimensional SciDB binary format file and push the data on top of the present array. Metadata, such as timestamps and meaning of the pixel values, are persisted in an additional file-based database. Standardized access to the data is facilitated via Web Coverage and Web Map Services (WCS and WMS by the OGC). We support common outputformats for geo-spatio-temporal datasets such as netCDF and GeoTIFF. While the WCS interface provides access to the raw data used in subsequent models and analyses, the WMS provides styled images that can easily be added to map clients. Internally, the access organised through a Web Processing Service that evaluates the WCS and WMS request and can be enhanced with additional (pre-)processing functionality. These implementations are made within the MuDak-WRM project, where a particular goal is the development of a central data delivering service for EO scenes and in-situ measurements at water reservoirs. The quality of a water reservoir does not only depend on the reservoir itself, but is influenced by the surrounding environment and the entire catchment. An in-situ monitoring of a reservoir is detailed, but also very costly. Hence, the MuDak-WRM project aims at identifying proxies that provide insights into the quality of a reservoir detailed enough for a mid-range management of the reservoir that are applicable worldwide. The MuDak-WRM project is funded by the German Federal Ministry of Education and Research (BMBF) with in the GRoW funding measure.