Data-driven Modeling, Prediction and Predictability: The Complex Systems Framework

The data-driven modeling of dynamics, spatial complexity and fluctuations in large-scale open systems in nature using observational data have yielded many significant advances. This approach is based on the dynamical systems theory which provides a framework with proper mathematical foundation for developing models that embody the features of the system inherent in the data, independent of modeling assumptions. This key feature is enabled by the recognition that the dynamics of nonlinear dissipative systems evolves in a phase space with a limited number of degrees of freedom, thus only a small number of variables are essential to model the dynamics. An early applications of this data-driven modeling is the first predictions of space weather, which used data from ground- and space-based measurements. Along with predictions, quantifying the predictability has been a long-standing challenge, in particular for extreme events. The recognition that an ensemble of similar initial states will undergo a spreading as they approach extreme situation has led to a new technique for prediction of extreme events. An ensemble transform Kalman filter (ETKF) technique developed for a dynamical system whose phase space is reconstructed from data, with unknown model equations, can be used to investigate the nature of the ensemble spread. In a study of extreme space weather, the ETKF was used to follow the trajectories of an ensemble of states in the phase space reconstructed from the time series data of the auroral elctrojet index AL. The ensemble spread was found to vary proportional to the AL intensity, thus providing a potential precursor to an extreme event. In weather and climate, there has been a growing realization of the need for prediction beyond weather, placing the predictability at intraseasonal and seasonal time scales as a key problem. This was addressed using extensive data of Indian monsoon rainfall to develop a dynamical model in the reconstructed phase space. An analysis of the evolution of many initial states showed predictability at intraseasonal time scale, and comparisons with global climate models lead to identification of ways to improve their predictive capability.