What can decision makers achieve from computer simulations of environmental systems?

For scientists and decision-makers to understand model predictions and their limitations, models need to be as transparent and refutable as possible. This is achieved by evaluating model fit to data, estimated parameter values, sensitivities, and uncertainty. This talk illustrates methods for evaluating model accuracy, identifying important parameters and observations, quantifying uncertainty, and identifying potential new observations. We also point out some important challenges. First, advances in computing power notwithstanding, computational runtimes remain a major constraint as environmental models become more complicated in an attempt to better capture realistic complexity, heterogeneity and non-stationarity. This constraint is often particularly restrictive given the continuing push towards computationally intensive analysis methods requiring 10,000s or more model runs. In environmental fields, where models can take a week or more per forward run, such methods are burdensome and often infeasible. Second, the relationships between the various model analysis methods and metrics in current use and in research are yet to be clearly established. This makes it difficult for research managers - and even researchers themselves - to develop strategic insights from the enormous ongoing effort to model environmental systems. In our strategy for navigating these difficulties, we suggest viewing the plethora of methods and metrics based on their objectives and computational demand, and making clear links between methods pursuing the same objectives despite starkly different theoretical backgrounds. The strategy emphasizes practical solutions as embodied in the proposed integrated use of methods that range from being computationally frugal (typically local) to demanding (typically global). We identify inexpensive diagnostics to distinguish between cases where frugal methods provide adequate and efficient insights into complex, high-dimensional models and enable systematic comparison of many alternative models and hypotheses, versus cases where more general but less computationally efficient methods are needed. Applied systematically, such an approach will eventually provide the insight needed to model environmental systems more strategically, better serving the needs of resource managers and public policy.