The Terra Data Fusion Project
Abstract
Terra is the flagship of NASA's Earth Observing System. Launched in 1999, Terra's five instruments continue to gather data that enable scientists to address fundamental Earth science questions. By design, the strength of the Terra mission has always been rooted in its five instruments, as well as the ability to fuse their data together for obtaining greater quality of information for Earth Science compared to individual instruments alone. As the data volume grows and the central Earth Science questions move towards problems requiring decadal-scale data records, the need for data fusion and the ability for scientists to perform large-scale analytics with long records have never been greater. The challenge is particularly acute for Terra, given its growing volume of data (> 1 petabyte), the storage of different instrument data at different archive centers, the different file formats and projection systems employed for different instrument data, and the inadequate cyberinfrastructure for scientists to access and process whole-mission fusion data (including Level 1 data). Sharing newly derived Terra products with the rest of the world also poses challenges. As such, the Terra Data Fusion Project aims to resolve two long-standing problems: 1) How do we efficiently generate and deliver Terra data fusion products? 2) How do we facilitate the use of Terra data fusion products by the community in generating new products and knowledge through national computing facilities, and disseminate these new products and knowledge through national data sharing and cloud services? Here we will provide an update on significant progress made in addressing these problems by working with NASA and NCSA, including the development of a new L1B Terra Basic Fusion product, new Advanced Fusion tools, and performance metrics on HPC and cloud systems. Results from several science-specific drivers for Terra fusion products will also be presented. We demonstrate that the Terra Data Fusion Project itself provides an excellent use-case for the community addressing Big Data and cyber infrastructure problems.
- Publication:
-
AGU Fall Meeting Abstracts
- Pub Date:
- December 2018
- Bibcode:
- 2018AGUFMIN43A..12D
- Keywords:
-
- 0399 General or miscellaneous;
- ATMOSPHERIC COMPOSITION AND STRUCTUREDE: 3399 General or miscellaneous;
- ATMOSPHERIC PROCESSESDE: 1899 General or miscellaneous;
- HYDROLOGYDE: 1996 Web Services;
- INFORMATICS