ECHO - Search and Order Metadata Registry Post EDG to WIST Transition
Abstract
NASA’s EOS Data and Information System (EOSDIS) seeks to support the data and information management system services that are required to further NASA’s Science Missions Directorate (SMD) mission. This mission is to develop a scientific understanding of the Earth system and its response to natural and human-induced causes. Two of EOSDIS central components is the EOSDIS Core System (ECS) and its EOS Clearing House (ECHO).ECHO reached a new level of operational demands in 2008, when the legacy EOS Data Gateway (EDG) client and related components were retired in favor of a new paradigm for search and order . Under this new approach, searches over the 2.5+ petabytes of NASA remote sensing images can be done by querying the associated metadata held in ECHO’s database. Since the completion of the EDG retirement in Feb 2009 ECHO has assumed an enhanced operational posture, meeting or surpassing all former EDG performance, capabilities and levels of usage. This operational readiness is demonstrated by a number of relevant metrics as outlined in the following paragraphs. The first data center to retire their EDG instance for use of ECHO’s web-based search-and-order client, Warehouse Inventory Search Tool (WIST), was the National Snow and Ice Data Center (NSIDC) on August 2008. Land Processes LPDAAC, ECHO’s biggest data provider, followed in September that same year. The last provider (Langley Research Center - LaRC) completed its migration on February. 2009. During this transition period over 5000 EDG registered users were migrated successfully by deploying a custom tool that allowed the seamless replication of EDG accounts into the ECHO system. Since the retirement of EDG, ECHO has seen a steady increase of users at a rate of over 100 new users per week on average. As providers migrated out of the legacy system, ECHO saw a steady increase in orders and searches. Within a couple of months after the LPDAAC migration, ECHO had absorbed all of the legacy usage (unclear what is meant by this), both in searches and orders. Currently, ECHO serves an average of over 25,000 queries and 2,500 orders on a weekly basis. Through targeted architectural changes and continued software tuning, ECHO has improved its performance metrics in a significant way. Average query performance has been more than halved since the beginning of 2009 and the standard deviation for queries has been reduced to an eighth in the same period. The increased system performance capability was highlighted during the first week of the ASTER GDEM June 2009 release by sustaining increased performance demands and higher rates of user service levels. This was seen through an increase of four times the demand for orders, 10 times the number of searches and more than double the number of registered users. The excellent response of the ECHO system and team is the result of months of focused and continuous improvements in ECHO’s performance, availability and stability. With availability metrics above 98% the ECHO system demonstrates full operational maturity; ready and available to fulfill NASA Science Missions Directorate mission of making EOSDIS data broadly available and usable by a variety of science applications.
- Publication:
-
AGU Fall Meeting Abstracts
- Pub Date:
- December 2009
- Bibcode:
- 2009AGUFMIN41A1107B
- Keywords:
-
- 1600 GLOBAL CHANGE;
- 1908 INFORMATICS / Cyberinfrastructure;
- 1946 INFORMATICS / Metadata;
- 1996 INFORMATICS / Web Services