Machine learning for predicting the Bz magnetic field component from upstream in situ observations of solar coronal mass ejections

Predicting the Bz magnetic field embedded within ICMEs, also known as the Bz problem, is a core problem in space weather research and prediction. We shine new light on the Bz problem by entertaining the hypothesis that upstream in situ measurements of the ICME sheath region and the first few hours of the magnetic obstacle provide sufficient information for predicting the downstream Bz component. To do so, we present a predictive tool based on machine learning that is trained and tested on 348 ICMEs observed in Wind, STEREO-A, and STEREO-B mission data and extracted from the open-source ICMECATv2.0 catalog. To challenge the predictive tool, based on random forests, in an experimental real-time scenario, we let the ICMEs sweep over the spacecraft and assess how continually feeding new in situ data into machine learning enhances the Bz prediction. Because any scenario to use the predictive tool in operational space weather forecasting needs a reliable automated ICME identification functionality, we examine the robustness of an existing automated ICME identification algorithm for the time intervals under scrutiny. We find that the predictive tool can predict estimates of the Bz component in the magnetic obstacle of an ICME in reasonable agreement with observations. While this study is unlikely to solve the Bz problem, the predictive tool shows promise for ICMEs with a recognizable magnetic flux rope signature. In this presentation, we will discuss the main challenges we face in using a data-driven machine learning application to solve one of the vital problems in space weather forecasting. We will outline the lessons learned and future strategies for predicting and potentially mitigating the effects of ICMEs on our planet Earth and its inhabitants.