Insights on Emulation Performance from Large Ensemble Experiments

Taking advantage of a set of simulations with CESM1-CAM5 under a range of scenarios, each comprising multiple members with varying initial conditions, we test the performance of well established emulation techniques, like pattern scaling and time-shift, for a suite of extreme indicators from the ETCCDI list. These techniques have been long tested and applied for either single model projections or multi-model ensembles, but have less frequently been tested taking advantage of sizable initial condition ensembles from a single model.

The role of internal variability in affecting the performance evaluation is brought to the fore in our analysis, thanks to the availability of multiple initial condition ensembles. In particular, a decomposition of the error allows to measure the contribution of internal variability (in either the true or the emulated quantities) as opposed to the structural emulation error. A comparison of the relative importance of these three components across the suite of indices offers insights in the degree of challenge that the different metrics of extreme behavior pose to simple emulation techniques. Differences in the relative importance of these components also surface across space, when the evaluation is done at a geographically disaggregated scale, highlighting regions that pose particular challenges to the emulation of extremes, or, on the other hand, where the effect of internal variability is especially large.