Enhancing Community Collaborations through NWP Software Containers
Abstract
A major hurdle when attempting to run a modeling system is properly setting up and compiling all of the necessary code components, including a number of external libraries. In addition to running a forecast model, users often need pre- and post-processing software, as well as a way to visualize and verify output. To ease the burden of establishing an end-to-end workflow from scratch, the concept of containers has been gaining momentum in the modeling community. Containers allow for technical advantages such as easily bundling all necessary components of end-to-end software systems and allowing for highly portable/sharable code as well as scientific applications such as reproducibility. NCAR colleagues have established containers to run a subset of an end-to-end NWP system, including the Weather Research and Forecasting (WRF) Pre-Processing System (WPS), WRF model, and the NCAR Command Language (NCL). To provide additional capabilities, the Developmental Testbed Center (DTC) has developed containers for the Unified Post-Processor (UPP), Model Evaluation Tools (MET), and the METViewer database and display software systems. Through this complementary work, a full end-to-end NWP system has been established for two case studies, with ongoing work to containerize the Gridpoint Statistical Interpolation (GSI) data assimilation system. The DTC has also created a comprehensive online tutorial that guides users through building and running the case studies; the tutorial also includes helpful instructions for adapting the containers for customized needs. With models being run at higher resolution and as the use of ensembles become more prevalent, the necessary computational resources and increasing size of data volumes is making it natural to explore running containerized end-to-end NWP systems on high performance computing (HPC) and/or cloud computing platforms. In addition, cloud computing offers an excellent opportunity to provide a sandbox to the community to share output and datasets, allowing for cross-community interactions. In the near future, the DTC will be transitioning their NWP containers to be functional on HPC and cloud computing platforms. This presentation will highlight the motivation for using NWP containers and provide a status update on running the DTC end-to-end NWP containers in the cloud.
- Publication:
-
AGU Fall Meeting Abstracts
- Pub Date:
- December 2018
- Bibcode:
- 2018AGUFMIN53A..05H
- Keywords:
-
- 1908 Cyberinfrastructure;
- INFORMATICSDE: 1916 Data and information discovery;
- INFORMATICSDE: 1948 Metadata: Provenance;
- INFORMATICSDE: 1976 Software tools and services;
- INFORMATICS