Binomial model

In a large class of experiments, the observations consist of counts, that is, a number of things (events, occurrences, etc.). In many processes of physics interests the resulting number of counts is described probabilistically by a binomial or a Poisson model. For example, we want to draw an inference about the efficiency of a detector, a branching ratio in a particle decay or a rate from a measured number of counts in a given interval of time.

The binomial distribution describes the probability of randomly obtaining $n$ events (`successes') in $N$ independent trials, in each of which we assume the same probability $\theta$ that the event will happen. The probability function is

$$p(n\,\vert\,\theta,N) = \left(\!\!\begin{array}{c}N \\ n\end{array}\!\!\right) \theta^n(1-\theta)^{N-n}\,,$$ (37)

where the leading factor is the well-known binomial coefficient, namely ${N!}/{n!(N-n)!}$. We wish to infer $\theta$ from an observed number of counts $n$ in $N$ trials. Incidentally, that was the ``problem in the doctrine of chances'' originally treated by Bayes (1763), reproduced e.g. in (Press 1992). Assuming a uniform prior for $\theta$, by Bayes' theorem the posterior distribution for $\theta$ is proportional to the likelihood, given by Eq. (37):

$$p(\theta \,\vert\,n,N,I) = \frac{ \theta^n\,(1-\theta)^{N-n}} {\int_0^1 \theta^n\,(1-\theta)^{N-n}\,\mbox{d}\theta}$$ (38)

$$= \frac{(N+1)!}{n!\,(N-n)!}\,\theta^n\,(1-\theta)^{N-n}\,.$$ (39)

Figure 1: Posterior probability density function of the binomial parameter $\theta$, having observed $n$ successes in $N$ trials.

Some examples of this distribution for various values of $n$ and $N$ are shown in Fig. 1. Expectation, variance, and mode of this distribution are:

$$\mbox{E}(\theta) = \frac{n+1}{N+2}$$ (40)

$$\sigma^2(\theta) = \frac{(n+1)(N-n+1)}{(N+3)(N+2)^2} = \frac{\mbox{E}(\theta)\,\left(1 - \mbox{E}(\theta)\right)}{N+3}$$ (41)

$$\theta_{\mbox{\footnotesize m}} = \frac{n}{N}\,,$$ (42)

where the mode has been indicated with $\theta_{\mbox{\footnotesize m}}$. Equation (40) is known as the Laplace formula. For large values of $N$ and $0 \ll n \ll N$ the expectation of $\theta$ tends to $\theta_{\mbox{\footnotesize m}}$, and $p(\theta)$ becomes approximately Gaussian. This result is nothing but a reflection of the well-known asymptotic Gaussian behavior of $p(n\,\vert\,\theta,N)$. For large $N$ the uncertainty about $\theta$ goes like $1/\sqrt{N}$. Asymptotically, we are practically certain that $\theta$ is equal to the relative frequency of that class of events observed in the past. This is how the frequency based evaluation of probability is promptly recovered in the Bayesian approach, under well defined assumptions.

Figure 2: The posterior distribution for the Poisson parameter $\lambda$, when $n$ counts are observed in an experiment.