Unified approach to the classical statistical analysis of small signals
Abstract
We give a classical confidence belt construction which unifies the treatment of upper confidence limits for null results and twosided confidence intervals for nonnull results. The unified treatment solves a problem (apparently not previously recognized) that the choice of upper limit or twosided intervals leads to intervals which are not confidence intervals if the choice is based on the data. We apply the construction to two related problems which have recently been a battleground between classical and Bayesian statistics: Poisson processes with background and Gaussian errors with a bounded physical region. In contrast with the usual classical construction for upper limits, our construction avoids unphysical confidence intervals. In contrast with some popular Bayesian intervals, our intervals eliminate conservatism (frequentist coverage greater than the stated confidence) in the Gaussian case and reduce it to a level dictated by discreteness in the Poisson case. We generalize the method in order to apply it to analysis of experiments searching for neutrino oscillations. We show that this technique both gives correct coverage and is powerful, while other classical techniques that have been used by neutrino oscillation search experiments fail one or both of these criteria.
 Publication:

Physical Review D
 Pub Date:
 April 1998
 DOI:
 10.1103/PhysRevD.57.3873
 arXiv:
 arXiv:physics/9711021
 Bibcode:
 1998PhRvD..57.3873F
 Keywords:

 06.20.Dk;
 14.60.Pq;
 Measurement and error theory;
 Neutrino mass and mixing;
 Physics  Data Analysis;
 Statistics and Probability;
 High Energy Physics  Experiment
 EPrint:
 40 pages, 15 figures. Changes 15Dec99 to agree more closely with published version. A few small changes, plus the two substantive changes we made in proof back in 1998: 1) The definition of "sensitivity" in Sec. V(C). It was inconsistent with our actual definition in Sec. VI. 2) "Note added in proof" at end of the Conclusion