Evaluating Earthquake Models and Forecasts Through CSEP

Two of the primary goals of the Collaboratory for the Study of Earthquake Predictability (CSEP) are (1) reducing the controversy surrounding earthquake prediction and (2) promoting rigorous research on earthquake predictability. An essential part of achieving these goals is rigorous and transparent testing and evaluation of submitted earthquake forecasts. Many types of tests have been adopted by CSEP, but the list is neither exhaustive nor fully understood. Repeating the CSEP calculations and performing a suite of additional tests, we analyzed the forecasts submitted to the 2005 RELM experiment to better determine which aspects of the forecasts were being evaluated by each test. We summarize our findings by listing the strengths and weaknesses of each test and suggest additional tests to be incorporated into the CSEP framework.