Towards Digital Twins for NVIDIA's Earth-2 Initiative: Pushing the Limits of Deep Auto-regressive Fourier Neural Operator and Transformer models for Earth System Emulation
Abstract
NVIDIA is committed to helping the scientific community address climate change. In Nov. 2021, our CEO announced the Earth-2 initiative, which aims to build digital twins of the Earth and a dedicated supercomputer, E-2, to power them. Two central goals of this initiative are: (i) Computational: enable high-resolution hybrid climate-ML predictions with credible cloud physics via NVIDIA Modulus; and (ii) Societal: nimbly serve interactive, high-fidelity, high-resolution climate predictions via NVIDIA Omniverse.
Next-generation km-scale climate simulations are prohibitively expensive and produce unmanageable data volumes. Therefore, the above-mentioned goals depend on achieving orders-of-magnitude speedup and data compression via "tethering" a skillful ML surrogate to checkpoints of km-scale accelerated hybrid ML-climate models, proposed by Bjorn Stevens [Digital Twins for Understanding and Adapting to Climate Change] at GTC Spring 2022 as part of NVIDIA's four-part climate simulation strategy, "Hop. Skip. Tune. Leap." In this context, we present the latest results from FourCastNet, a sophisticated, transformer-based adaptive Fourier Neural Operator (FNO) deep learning model for auto-regressive forecasting. While ultimately intended for use in climate, its architecture and predictive skill limits are being refined in the context of global high-resolution weather prediction. We present promising skill scores approaching the ECMWF IFS gold standard achieved from ambitious re-trainings on: (i) increasingly larger fractions of the full ERA5 reanalysis state vector; (ii) expanding FNO from 2D to 3D to account for the vertical structure of the atmosphere and ocean; and (iii) expanding the time stencil from 12-hour to larger time horizons, to incorporate temporal history and correlations. We report on experiments in a range of latent space noise injection strategies to achieve increasingly realistic chaotic trajectory divergence and to calibrate large forecast ensembles. We present preliminary results on regionalizing forecasts with high-resolution models nested within global FourCastNet. Finally, we discuss the implications of skill transfer across datasets and initializations generated by different Numerical Weather Prediction (NWP) models for real-time forecasting.- Publication:
-
AGU Fall Meeting Abstracts
- Pub Date:
- December 2022
- Bibcode:
- 2022AGUFMGC16C..05K