Dos and don'ts of reduced chisquared
Abstract
Reduced chisquared is a very popular method for model assessment, model comparison, convergence diagnostic, and error estimation in astronomy. In this manuscript, we discuss the pitfalls involved in using reduced chisquared. There are two independent problems: (a) The number of degrees of freedom can only be estimated for linear models. Concerning nonlinear models, the number of degrees of freedom is unknown, i.e., it is not possible to compute the value of reduced chisquared. (b) Due to random noise in the data, also the value of reduced chisquared itself is subject to noise, i.e., the value is uncertain. This uncertainty impairs the usefulness of reduced chisquared for differentiating between models or assessing convergence of a minimisation procedure. The impact of noise on the value of reduced chisquared is surprisingly large, in particular for small data sets, which are very common in astrophysical problems. We conclude that reduced chisquared can only be used with due caution for linear models, whereas it must not be used for nonlinear models at all. Finally, we recommend more sophisticated and reliable methods, which are also applicable to nonlinear models.
 Publication:

arXiv eprints
 Pub Date:
 December 2010
 arXiv:
 arXiv:1012.3754
 Bibcode:
 2010arXiv1012.3754A
 Keywords:

 Astrophysics  Instrumentation and Methods for Astrophysics;
 Physics  Data Analysis;
 Statistics and Probability;
 Statistics  Methodology
 EPrint:
 12 pages, 3 figures, 1 table