CMIP6 Evaluation with the ESMValTool

In recent years the scope of model evaluation has expanded dramatically, with new insightful approaches being added to traditional methods that remain important and therefore are frequently applied to any new simulation. This therefore is an opportunity to automatize well-established aspects of model evaluation so that they can be made available more quickly, with less effort, and can be used as a resource to model analysts and developers. To accomplish this in time for utilization in the Coupled Model Intercomparison Project (CMIP) Phase 6, a new version of the Earth System Model Evaluation Tool (ESMValTool, https://www.esmvaltool.org/) has been developed as open-source software by more than 41 institutions. The ESMValTool has undergone major improvements since its first release in 2016 and is now a well-tested tool that provides end-to-end provenance tracking to ensure traceability. It consists of a Python backend that uses the Iris library to efficiently perform common pre-processing operations and a diagnostic part that includes tailored diagnostics and performance metrics for specific scientific applications. ESMValTool currently supports diagnostics written in Python, NCL, R and Julia. The diagnostic part includes a large collection of standard recipes for reproducing the analysis of many variables across atmosphere, ocean, and land domains, with diagnostics and performance metrics focusing on the mean-state, trends, variability and important processes, phenomena, as well as emergent constraints. These diagnostics are partly based on model-to-model comparisons, but many also use observational data sets (satellite and ground-based observations) for model evaluations. Additionally, detailed diagnostics for monsoon, El Nino Southern Oscillation (ENSO), and the Madden-Julian Oscillation (MJO) are included, as well as the majority of analyses from the Intergovernmental Panel on Climate Change (IPCC) AR5 climate model evaluation chapter which allows comparison of the new CMIP6 simulations as they are published to the Earth System Grid Federation (ESGF). A growing list of more targeted analysis packages, such as the NCAR Climate Variability Diagnostics Package (CVDP), the MetOffice Auto-assess package, and the Barcelona Supercomputing Center s2dverification package can also be used from ESMValTool.