End-to-End Workflow-Driven Hydrologic Analysis for Different User Groups in HydroFrame
Abstract
The HydroFrame project is a community platform designed to facilitate integrated hydrologic modeling across the US. As a part of HydroFrame, we seek to innovate workflows and provide new pathways that enable hydrologic analysis for three target user groups: the modeler, the analyzer, and the domain science educator.
The 'modeler' is mostly likely a hydrologic scientist or an engineer who has advanced knowledge of using integrated hydrologic models for their research but lack the background or computational resources to run and develop complex models on their own. An 'analyzer' makes use of a set of existing simulation outputs but do not want to interact with the model and generate custom outputs. This user could be a hydrologist or any other domain scientist who is looking for high resolution spatial datasets of hydrologic fluxes, and/or interested in interacting with the software framework similar to other static model output repositories (e.g., the IGBP global climate model outputs). The simulation outputs also represent a great and unrealized resource for education outreach at many levels. The third user group, 'the domain science educator', would like to use the simulations to create educational tools. This group could include K-12 educators teaching students about the hydrologic cycle and their local watershed. We present the initial progress on the HydroFrame community platform using an automated Kepler workflow that performs end-to-end hydrology simulations involving data ingestion, preprocessing, analysis, modeling, and visualization. We will demonstrate how different modules of workflow can be reused and repurposed for the three target user groups. Moreover, the Kepler workflow ensures complete reproducibility through a built-in provenance framework that collects workflow specific parameters, software versions and hardware system configuration. The poster will also present a design to leverage provenance data and machine learning techniques to predict performance and forecast failures ahead of time using an automatic performance collection component of the pipeline, with a goal to optimize utilization of large-scale computational resources to adjust to the needs of all three user groups.- Publication:
-
AGU Fall Meeting Abstracts
- Pub Date:
- December 2019
- Bibcode:
- 2019AGUFM.A13H3033P
- Keywords:
-
- 3336 Numerical approximations and analyses;
- ATMOSPHERIC PROCESSES;
- 3337 Global climate models;
- ATMOSPHERIC PROCESSES