Analysis and Review of NASA Earth Science Metadata: How Automation Plays a Role

In 2015, the NASA-developed Common Metadata Repository (CMR) was implemented to consolidate NASA's Earth science metadata into a single authoritative repository. In order for the CMR to maintain a high standard of quality, the Analysis and Review of CMR (ARC) team at NASA Marshall Space Flight Center is manually curating metadata records provided to the CMR by EOSDIS's twelve Distributed Active Archive Centers (DAACs). The methodology of this curation involves an evaluation and assessment of all metadata records, both collection and granule level, followed by improvement recommendations, which are submitted to the DAACs for correction and implementation.

A key tool now used in the curation process is an online curation dashboard developed in collaboration with a software development company, Element 84. This tool facilitates the review of Earth science metadata records and subsequent stakeholder collaboration on the resolution of identified issues. A key capability of the new tool is a suite of automated compliance checks written in Python that verify the integrity of various metadata elements across multiple dialects. For some elements, the ARC team is only concerned with the presence of a value; whereas other elements need to be scrupulously validated against an EOSDIS dialect-specific schema. The automated compliance checks include the testing of logical collection-granule relationships, the handling of URL HTTPS response codes, the validation of controlled keywords, and more. This presentation will focus on the usage of Python scripts and methods of use in the dashboard tool.