Model Validation Tool: Comprehensive Assessment of Models and Events using Library Tools (CAMEL)

The Comprehensive Assessment of Models and Events using Library Tools (CAMEL) framework leverages existing Community Coordinated Modeling Center (CCMC) services: Run-on-Request post-processing tools that generate model time series outputs and the new CCMC Metadata Registry (CMR) that describes simulation runs using SPASE metadata. The new CAMEL visualization tool compares the modeled time series with observational data and computes a suite of skill scores such as Prediction Efficiency, Root Mean Square Error, and Symmetric Signed Percentage Bias. Model-data pairs used for skill calculations are obtained considering a user-selected maximum difference between the time of observation and the nearest model output. The system renders available data for all locations and time periods selected using interactive visualizations that allow the user to zoom, pan and pick data values along traces. Skill scores are reported for each selected event or aggregated over all events for all participating model runs. Separately, scores are reported for all locations (satellites or stations) and for each location individually. We are building on past experiences with model-data comparisons of magnetosphere and ionosphere model outputs from GEM2008, GEM-CEDAR ETI and the SWPC Operational Space Weather Model challenges. The CAMEL visualization tool is demonstrated using three validation studies: a) Wang-Sheeley-Arge (WSA) heliosphere simulations compared against OMNI solar wind data, b) ground magnetic perturbations from several magnetosphere and ionosphere electrodynamics models as observed by magnetometers, and c) electron fluxes from several ring current simulations compared to RBSP HOPE instrument measurements, integrated over different energy ranges.