Distributed Modeling on Demand: An Open Source Infrastructure for Facilitating Exploration and Usage of Computational Models

Computational environments serve as the substrate for which models operate. Such environments range in form from simplistic to complex both in underlying architecture (i.e. solvers) and interface (i.e. UI) as defined by their users needs. As a result, setup, usage, and maintenance of aforementioned environments are time consuming tasks that require adequate technical knowledge. Thus, these often become barriers that hinder model and modeling ecosystem research and development efforts. The Office of Water Prediction's (OWP) Distributed Modeling on Demand (DMOD) project seeks to reduce the cognitive load on users and allow them to run and explore model outputs with a single dependency. In this project we have built a prototype user-friendly automated infrastructure that allows a modeling system to deploy across many environments with minimum knowledge of the systems. By abstracting computational resources, we virtually manage the availability, addition, and subtraction of hardware resources, allowing the environment to flexibly adapt to a given deployment using service discovery techniques. Containerized input (forcing) and output handling allow customizations to run arbitrary models and tasks in this framework, and in some cases default handlers can be re-used by models, allowing quick integration into the framework. Integrated models benefit from automatic job provisioning and multiple resource allocation methods for either batch or distributed (parallel) jobs. Jobs are automatically monitored by the system and can be recovered from known error conditions. This same monitoring also allows users to view metrics like running time and the position in the queue. We present the preliminary support of multiple models and workflows using DMOD to accelerate and complement OWP's various modeling objectives and research-to-operations pipeline.