Big Data Challenges Indexing Large-Volume, Heterogeneous EO Datasets for Effective Data Discovery

This paper describes the importance and challenges faced in making Earth Observation datasets discoverable and accessible by the widest possible user base. Concentrating on data discovery, it details work that is being undertaken by the Centre for Environmental Data Analysis (CEDA), to ensure that the datasets held within its archive are discoverable and searchable. One aspect of this is in indexing the data using controlled vocabularies, based on a Simple Knowledge Organization System (SKOS) ontology, and hosted in a vocabulary server, to ensure that a consistent understanding and approach to a faceted search of the data can be achieved via a variety of different routes. This approach will be illustrated using the example of the development of the ESA CCI Open Data Portal.