Improving Methods for Selecting and Evaluating Earth System Models for Climate Change Impact Applications over the Pacific Northwest

The recent completion of most Coupled Model Intercomparison Project Phase 6 (CMIP6) experiments necessitates new methods to evaluate the performance and suitability of earth system models (ESMs) for selecting and weighting ESMs for hydroclimate applications. The extremely large data volumes now present in the CMIP experiments could greatly complicate nascent stakeholder efforts to use new ESM outputs in their updated climate vulnerability and impact assessments. This is due to the combinatorial effect of including multiple models and analysis methods at each link in the impact modeling chain (ESMs - Downscaling models - Impact models), which is becoming a significant point of concern. Without sub-selecting ESMs in a clear and rational way, the newest science contained in the ESMs may not be available for impacts modelers to improve their own models and results. That, in turn, would severely limit the ability of water and energy sector managers to see and understand climate threats to their planning and operations, and to enhance the resilience of their plans and operations against those threats.

To begin to address these two core issues, the National Center for Atmospheric Research and the US Army Corps of Engineers (USACE) have embarked on a project to co-design methods for ESM selection and weighting. First, we develop a suite of metrics applied over the Pacific Northwest (PNW), including a variety of process-oriented metrics and more traditional metrics such as trend analysis and spatial correlation of climatological means, as well as metrics such as frequency of superior performance. These metrics are compared with observational and reanalysis datasets to estimate the observational envelope in comparison with that of CMIP6 historical runs. We also perform several more traditionally short-term forecast verification concepts. For example, model performance changes between split samples (e.g., 1950-1980 versus 1990-2020), and concepts such as pairwise differences to increase the power of discriminating between models. The models are then ranked relative to each other and the effect of rank-based weighting or culling on the projected trend envelopes of precipitation and temperature over the next century are compared via the Shared Socioeconomic Pathways (SSP) runs of the CMIP6 models.