Fast GP Estimation for In-situ Inference of Large Scale Climate Modeling with Statistical Machine Learning
Abstract
Complex models for the predictions of climate and environmental processes typically run on large supercomputers. However, such exascale computing are limited due to their I/O restrictions resulting in a gap between the complexity of the simulations run on such supercomputers and the amount of resulting data saved for post-hoc analysis. Statistical in-situ processing has evolved over the past decade as an alternative by fitting statistical models to data as the simulation runs. However, in-situ estimation of statistical models for exascale simulation data is difficult for many reasons as data sets are large, distributed, and can be streaming with communication band- width limitations. In addition, the estimation needs to be scalable and fast. Most models that are used to predict climate and environmental process are spatio-temporal, i.e., models that evolve their relevant state over some spatial domain (for e.g., the entire earth) for some prescribed time. In this work, we fit a spatial statistical model using Gaussian Process (GP) for the in-situ analysis of the E3SM model, the Department of Energy's exascale climate model. We developed a convolutional neural network (CNN) to predict the GP parameters for a spatial data set from a simulation run rather than optimize the parameters directly. We then extend this framework to account for the unstructured nature of spatial discretization that occur in the numerical simulation of this models (and similar models) by implementing a Graphical Convolutional Network (Graph CNN) to predict GP parameters of the underlying spatial process. This neural network scheme produces parameter estimates that compare well with standard methods such as maximum likelihood estimation in predictive performance but is obtained four orders of magnitude faster.
- Publication:
-
AGU Fall Meeting Abstracts
- Pub Date:
- December 2022
- Bibcode:
- 2022AGUFMNH12B..01P