Climate-model Ensemble Consistency Testing

Constant evolution and improvement of large scale climate simulation codes such as the Community Earth System Model (CESM) necessitate quality checks to verify the constitution of new climate simulations. The CESM Ensemble Consistency Test (CESM-ECT) was developed as a flexible but objective method for checking the statistical consistency between an accepted ensemble of climate simulations and new simulations created with updated code or within a new computational infrastructure. CESM-ECT utilizes a testing framework based on the popular technique of Principal Component Analysis (PCA) to determine whether a set of new simulations is statistically distinguishable from the established ensemble of climate simulations. The test has proven very powerful and of high practical use and is now part of the official CESM release. However, while CESM-ECT excels at detecting the presence of a discrepancy, systematically determining the root causes leading to a discrepancy has turned out to be a formidable challenge. We will discuss our ongoing efforts, and previous failures, in this quest.