Looking beyond physical models and machine learning: Novel insights into soil moisture dynamics using multi-scale Big Data geostatistics

The past six decades has seen an explosive growth in remote sensing data across air, land, and water dramatically improving predictive capabilities of physical models and machine-learning algorithms. Physical models, however, suffer from rigid parameterization and lead to incorrect inferences when little is known about the underlying physical process. ML models, conversely, sacrifice interpretation for enhanced prediction. On an interpretation-prediction spectrum, physical models lie on one end while ML algorithms fall on the other. Geostatistics lie somewhere in the middle, and are an attractive alternative for spatio-temporal inference in a data-driven setting. They do not have strong assumptions like physical models yet enable physical interpretation and uncertainty quantification.

In this work, we develop a novel multi-scale geostatistical algorithm which can combine massive remote sensing datasets at different spatio-temporal resolutions for enhanced understanding of the underlying physical processes. We apply the proposed algorithm combining soil moisture data from Soil Moisture Active Passive (SMAP) and Soil Moisture and Ocean Salinity (SMOS) with point data from U.S Climate Reference Network (USCRN) and Soil Climate Analysis Network (SCAN) across Contiguous US (CONUS) uncovering novel insights into soil moisture dynamics across scales. Using an underlying covariate-driven spatio-temporal process, the effect of dynamic and static physical controls—vegetation, rainfall, soil texture and topography—on soil moisture is quantified. We find that vegetation, rainfall and topography affect the mean soil moisture distribution across CONUS while soil texture determines the spatio-temporal covariance between soil moisture pixels. We successfully forecast 3-day soil moisture across CONUS for multiple spatio-temporal scales accompanied by uncertainty metrics. Finally, we discuss the potential applicability of the algorithm to future soil moisture missions and broader Earth-System processes.