Assessment of climate model performance from three generations IPCC assessment reports, on the sub-regional scale

A study was conducted to investigate the performance of climate models from the Intergovernmental Panel for Climate Change (IPCC) assessment reports on the scale (6ox6o) of the Gangetic West Bengal region of east India. Climatological mean current climate ( 1961-90) of seasonal rainfall and temperature over the domain were compared with the results from the models from the six modelling centres common to the second, third and fourth assessment reports (SAR, TAR and AR4 respectively), in order to judge to what extent these global models have improved on the regional scale. Metrics for model evaluation are not yet firmly established in the study, so in this paper we compare and contrast the results from a number of different statistics namely spatial correlation coefficient, mean bias, agreement index (d-index), mean absolute error (MAE), total RMSE, centred RMSE, and normalised error variance (NEV). The visual comparison of the observed and simulated seasonal cycles shows that the precipitation is less well represented than temperature. The overall impression seems to be an improvement of GCMs performance across generation. The improvement of models from SAR to TAR, TAR to AR4 and from SAR to AR4 is not uniform. On the basis of MII, an average of 57% and 70% of the models improved their performance in simulating current climate from SAR to TAR and from TAR to AR4 respectively. The overall improvement estimated through OMII indicates about 75% of models improved their skill from SAR to AR4 via TAR. The overall best performance on this regional scale in the AR4 is the Japanese model, MIROC, but the best model in terms of improving skill from SAR to AR4 is the United States model from GFDL. Correcting for topographic errors produced an overall improvement of spatial patterns and error statistics, but does not significantly alter the overall ranking of the models.