openEO: an open API for cloud-based big Earth Observation processing platforms

Today's cloud-based platforms for processing Earth Observation (EO) data come with strongly differing interfaces. This makes it not only difficult to compare offerings in terms of available data sets, processing capability and costs, but also to verify results by comparing them across different platforms. Also, data scientists using python, R or javascript would benefit from being able to execute functions written in one of these languages directly on the cloud platforms in a standardised manner. The H2020-funded project openEO (http://openeo.org/) has as its goal to develop an open API that makes it possible to directly interact with these platforms using one of these languages. We will show preliminary results, and discuss further development. Today, most EO cloud platforms host collections of EO image files. We argue that a file-agnostic access to EO imagery through a data cube view can boost usability of EO data. Data cubes in Earth Observation data refer to data representations where the spatial dimensions are complemented with other dimensions such as the temporal or spectral dimensions. Data cubes, even virtual ones, can be created in many ways. openEO will enable researchers to work on a "data cube view" of the EO imagery and directly filter, aggregate or map functions over dimensions of a cube (e.g. spatial, temporal, spectral) without being concerned about how the data in the processing platform is organised (by granule, as collections of GRIB or NetCDF files, as one or several arrays in an array database, etc.). It will also integrate raster and vector data cubes. The performance of a given operation exploiting the various dimensions of a data cube will depend on the type of operation and the underlying organisation of the data. openEO will ease the comparison of cloud platforms by securing that any user-code following the openEO API can be interpreted and executed on any cloud platform implementing the openEO API. Besides comparing processing capability, cost and performance, this will also help progress the discussion on reproducibility of cloud-based geoscientific computing.