Near Real-time Validation of the Magstar Magnetometer Array: Challenges, Requirements and Techniques

Ground-based magnetometers play an important role in understanding the hazards posed by geomagnetic storms to the power-grid infrastructure due to ground-induced currents. CPI currently operates the NSF-funded MagStar magnetometer array, which is distributed largely across the northern and eastern continental United States, from the Canadian border in Minnesota and upstate New York, to Texas and Virginia. These stations provide continuous real-time data streams of the local geomagnetic field at a 1Hz cadence. There is thus a tremendous need for data validation algorithms that validate magnetometer data with sufficient accuracy and in near real-time to provide timely hazard information.

We present the results of an anomaly detection algorithm that detects noise in the magnetometer data due to external environment disturbances such as vehicles, and is capable of operating in near real-time. We also present a comparison of techniques in spike detection and quantification of noise characteristics of magnetometers. These algorithms are deployed in the context of a data validation framework that incorporates the knowledge of the subject matter expert, since it requires an expert to distinguish true geophysical events from non-geophysical environmental conditions. We discuss challenges in ensuring consistency and reproducibility of data annotation schemes for representing data quality.