Can MHD be used to forecast substorms?

Magnetospheric substorms have been modeled using magnetohydrodynamic (MHD) simulations in numerous studies over the years. However, most efforts to study substorms with MHD have simulated only a single, or at most a handful, of substorms. No attempt has been made to test whether the ability of an MHD model to predict observed substorms can be reproduced consistently across numerous events. Such a demonstration would be a significant step toward an MHD-based substorm forecasting capability. In order to rigorously test whether substorm prediction is possible using MHD, we simulated a 1-month period (January 2005) using the Space Weather Modeling Framework (SWMF). However, a secondary challenge to validating a predictive model is the lack of a widely accepted, precise definition or procedure that can be used to identify substorms, either in model output or observational data. Part of the reason for this is that substorms can be observed using a variety of observational datasets and techniques, each of which has the potential to miss some substorms or to incorrectly identify unrelated processes as possible substorms. To address this, we developed a framework to combine substorm signatures from multiple datasets, in order to create a list of substorm onsets that have been independently identified by several methods. We use this framework to identify probable substorms in both the model output and in observational data from the same time period and test the skill of our model at predicting substorm onsets within a specified time window. To quantify the model performance we compute a range of metrics including the Heidke skill score, and estimate 95% confidence intervals to show that these values are statistically significant. Our preliminary results demonstrate that our MHD simulations have significant predictive skill with respect to substorm timing.