Synthetic Spectranomics: Deep learning of surface 3-D geometry, chemistry, and hyperspectra to inform next-generation land models

Machine learning, and in particular deep learning, has transformed our approach to Earth observation and systems modeling, or EOSM. Trained on near-to-remote sensing observations, physical models, or hybrids of both, machine learning may improve existing model formulations while creating new classes of models. Potential applications include data geolocalization and assimilation, model optimization, model emulation/surrogates, upsampling/downscaling, land-surface reconstruction for physics-based 4-D models, learning new governing equations for latent or difficult-to-simulate dynamics, simulating observing systems with realistic noise (hybrid network + GAN), inverse modeling, causal inference, and model code generation, among others. Full utilization of protected global inventory datasets is possible with privacy preserving networks, while biases and adversarial robustness must be addressed. One important application is observation synthesis, allowing land models to benefit from the increased spatial, temporal, and spectral/polarimetric resolution proposed for future observing systems. This provides the continuous historical record needed to constrain, calibrate/validate, and develop models. Despite the many datasets released for remote sensing computer vision challenges to date, large labeled datasets remain scarce for a number of tasks relevant to Earth systems and planetary science generally. To address this limitation, we leverage complementary coincident observations from air- and space-borne platforms to generate labeled data relevant to modeling tasks. We focus on the information-theoretic task of maximizing mutual information between predictors X and targets Y , or feature extraction. First, we will utilize airborne spectranomics observations from NASA G-LiHT, NEON AOP, and/or ASU GAO to learn canopy geometry, chemistry, and hyperspectral reflectances from pseudo-multispectral and/or RGB imagery. Second, we aim to utilize NASA EO-1 Hyperion and ALI observations to learn the mapping from multi- to hyper-spectral. The trained model may then be applied to generate synthetic hyperspectral imagery using Landsat-8 OLI observations, for which ALI was a demonstrator. Subsequent work may extend this approach to commercial satellite constellations.