NASA's Global Imagery Management System: TIE

NASA's Earth Observing System Data and Information System (EOSDIS)' Global Imagery Browse Services (GIBS) is a system that provides full resolution imagery from a broad set of Earth science disciplines to the public. Using well-accepted standard protocols such as the Open Geospatial Consortium (OGC) Web Map Tile Service (WMTS), GIBS delivers global imagery efficiently and responsively. Behind this service, lies The Imagery Exchange (TIE), a workflow data management solution developed at the Jet Propulsion Laboratory. TIE is an Open Archival Information System responsible for orchestrating the workflow for acquisition, preparation, generation, and archiving of imagery to be served by the GIBS' web mapping tile service, OnEarth. The workflow collects imagery provenance throughout a product's lifecycle by leveraging the EOS Clearing House (ECHO) and other long-term metadata repositories in order to promote reproducibility. Through this focus on metadata, TIE provides spatial and temporal searching capabilities such as an OpenSearch interface as well as facilitating the generation of metadata standards such as the OGC GetCapabilities. Designed as a scalable system, TIE's subsystems can scale-up or scale-down depending on the data volume it handles through the usage of popular open source technologies such as Apache Zookeeper and Grails. This presentation will cover the challenges and solutions to developing such a horizontally scalable data management system where science products are often varied with disparate provenance pertaining to source platforms and instruments, spatial resolutions, processing algorithms, metadata models and packaging specifications.