Probabilistic treatment of the uncertainty from the finite size of weighted Monte Carlo data
Abstract
Parameter estimation in HEP experiments often involves Monte Carlo simulation to model the experimental response function. A typical application are forwardfolding likelihood analyses with reweighting, or timeconsuming minimization schemes with a new simulation set for each parameter value. Problematically, the finite size of such Monte Carlo samples carries intrinsic uncertainty that can lead to a substantial bias in parameter estimation if it is neglected and the sample size is small. We introduce a probabilistic treatment of this problem by replacing the usual likelihood functions with novel generalized probability distributions that incorporate the finite statistics via suitable marginalization. These new PDFs are analytic, and can be used to replace the Poisson, multinomial, and samplebased unbinned likelihoods, which covers many use cases in highenergy physics. In the limit of infinite statistics, they reduce to the respective standard probability distributions. In the general case of arbitrary Monte Carlo weights, the expressions involve the fourth Lauricella function F_{D}, for which we find a new finitesum representation in a certain parameter setting. The result also represents an exact form for Carlson's Dirichlet average R_{n} with n > 0, and thereby an efficient way to calculate the probability generating function of the Dirichletmultinomial distribution, the extended divided difference of a monomial, or arbitrary moments of univariate Bsplines. We demonstrate the bias reduction of our approach with a typical toy Monte Carlo problem, estimating the normalization of a peak in a falling energy spectrum, and compare the results with previously published methods from the literature.
 Publication:

European Physical Journal Plus
 Pub Date:
 June 2018
 DOI:
 10.1140/epjp/i201812042x
 arXiv:
 arXiv:1712.01293
 Bibcode:
 2018EPJP..133..218G
 Keywords:

 Physics  Data Analysis;
 Statistics and Probability;
 Astrophysics  Instrumentation and Methods for Astrophysics;
 High Energy Physics  Experiment;
 Mathematics  Statistics Theory
 EPrint:
 Eur. Phys. J. Plus (2018) 133: 218