Binomially distributed observables

Let us assume we have performed $n$ trials and obtained $x$ favourable events. What is the probability of the next event? This situation happens frequently when measuring efficiencies, branching ratios, etc. Stated more generally, one tries to infer the ``constant and unknown probability''^5.6of an event occurring.

Where we can assume that the probability is constant and the observed number of favourable events are binomially distributed, the unknown quantity to be measured is the parameter $p$ of the binomial. Using Bayes' theorem we get

$$f(p\,\vert\,x,n,{\cal B}) = \frac{ f(x\,\vert\,{\cal B}_{n,p})\,f_\circ(p) }{ \int_0^1 f(x\,\vert\,{\cal B}_{n,p})\,f_\circ(p)\,\rm {d}p }$$

$$= \frac{ \frac{n!}{(n-x)!\,x!}\,p^x\,(1-p)^{n-x}\,f_\circ(p) }{ \int_0^1 \frac{n!}{(n-x)!\,x!}\,p^x\,(1-p)^{n-x}\,f_\circ(p)\,\rm {d}p }$$

$$= \frac{ p^x\,(1-p)^{n-x}}{\int_0^1 p^x\,(1-p)^{n-x}\,\rm {d}p}\, ,$$ (5.31)

where an initial uniform distribution has been assumed. The final distribution is known to statisticians as $\beta$ distribution since the integral at the denominator is the special function called $\beta$, defined also for real values of $x$ and $n$ (technically this is a $\beta$ with parameters $a=x+1$ and $b=n-x+1$). In our case these two numbers are integer and the integral becomes equal to $x!\,(n-x)!/(n+1)!$. We then get

$$f(p\,\vert\,x,n,{\cal B}) = \frac{(n+1)!}{x!\,(n-x)!}\,p^x\,(1-p)^{n-x}\,,$$ (5.32)

some example of which are shown in Fig.

Figure: Probability density function of the binomial parameter $p$, having observed $x$ successes in $n$ trials.

The expectation value and the variance of this distribution are:

$$[p] = \frac{x+1}{n+2}$$ (5.33)

$$(p) = \frac{(x+1)(n-x+1)}{(n+3)(n+2)^2}$$ (5.34)

$$= \frac{x+1}{n+2}\left(\frac{n+2}{n+2} -\frac{x+1}{n+2}\right)\frac{1}{n+3}$$

$$= [p]\,\left(1 - \mbox{E}[p]\right)\,\frac{1}{n+3}\,.$$ (5.35)

The value of $p$ for which $f(p)$ has the maximum is instead $p_m=x/n$. The expression E$[p]$ gives the prevision of the probability for the $(n+1)$-th event occurring and is called the ``recursive Laplace formula'', or ``Laplace's rule of succession''.

When $x$ and $n$ become large, and $0 \ll x \ll n$, $f(p)$ has the following asymptotic properties:

$$[p] \approx p_m=\frac{x}{n}\,,$$ (5.36)

$$(p) \approx \frac{x}{n}\,\left(1-\frac{x}{n}\right)\,\frac{1}{n} = \frac{p_m\,(1-p_m)}{n}\,,$$ (5.37)

$$\sigma_p \approx \sqrt{\frac{p_m\,(1-p_m)}{n}}\,,$$ (5.38)

$$p \sim {\cal N}(p_m, \sigma_p)\,.$$ (5.39)

Under these conditions the frequentistic ``definition'' (evaluation rule!) of probability ($x/n$) is recovered.

Let us see two particular situations: when $x=0$ and $x=n$. In these cases one gives the result as upper or lower limits, respectively. Let us sketch the solutions:

• x = n:
  

  $$f(n\,\vert\,{\cal B}_{n,p}) = p^n ,$$ (5.40)

  $$f(p\,\vert\,x=n,{\cal B}) = \frac{p^n}{\int_0^1p^n\,\rm {d}p} = (n+1)\, p^n,$$ (5.41)

  $$F(p\,\vert\,x=n,{\cal B}) = p^{n+1}\, .$$ (5.42)

  
  To get the $95\, \%$ lower bound (limit):
  

  $$F(p_\circ\,\vert\,x=n,{\cal B}) = 0.05\, ,$$

  $$p_\circ = \sqrt[n+1]{0.05}\, .$$ (5.43)

  
  An increasing number of trials $n$ constrain more and more $p$ around 1.

• x = 0:
  

  $$f(0\,\vert\,{\cal B}_{n,p}) = (1-p)^n ,$$ (5.44)

  $$f(p\,\vert\,x=0,n,{\cal B}) = \frac{(1-p)^n}{\int_0^1(1-p)^n\,dp} = (n+1)\, (1-p)^n ,$$ (5.45)

  $$F(p\,\vert\,x=0, n, {\cal B}) = 1 - (1-p)^{n+1}\, .$$ (5.46)

  
  To get the $95\, \%$ upper bound (limit):
  

  $$F(p_\circ\,\vert\,x=0,n,{\cal B}) = 0.95,$$

  $$p_\circ = 1 - \sqrt[n+1]{0.05}\, .$$ (5.47)

  
  

The following table shows the $95\, \%$ probability limits as a function of $n$. The Poisson approximation, to be discussed in the next section, is also shown.

	Probability level = $95\, \%$
$n$	$x=n$	$x=0$
	binomial	binomial	Poisson approx.
			( $p_\circ=3/n$)
3	$p\ge 0.47$	$p\le 0.53$	$p\le 1$
5	$p\ge 0.61$	$p\le 0.39$	$p\le 0.6$
10	$p\ge 0.76$	$p\le 0.24$	$p\le 0.3$
50	$p\ge 0.94$	$p\le 0.057$	$p\le 0.06$
100	$p\ge 0.97$	$p\le 0.029$	$p\le 0.03$
1000	$p\ge 0.997$	$p\le 0.003$	$p\le 0.003$

To show in this simple case how $f(p)$ is updated by the new information, let us imagine we have performed two experiments. The results are $x_1=n_1$ and $x_2=n_2$, respectively. Obviously the global information is equivalent to $x=x_1+x_2$ and $n=n_1+n_2$, with $x=n$. We then get

$$f(p\,\vert\,x = n,{\cal B}) = (n+1)\,p^n = (n_1+n_2+1)\,p^{n_1+n_2}\, .$$ (5.48)

A different way of proceeding would have been to calculate the final distribution from the information $x_1=n_1$,

$$f(p\,\vert\,x_1 = n_1,{\cal B}) = (n_1+1)\,p^{n_1}\, ,$$ (5.49)

and feed it as initial distribution to the next inference:

$$f(p\,\vert\,x_1 = n_1, x_2=n_2,{\cal B}) = \frac{p^{n_2} f(p\,\vert\,x_1=n_1, {\cal B})} {\int_{0}^{1}p^{n_2}f(p\,\vert\,x_1=n_1, {\cal B})\,\rm {d}p}$$ (5.50)

$$= \frac{p^{n_2}(n_1+1)\,p^{n_1}} {\int_{0}^{1}p^{n_2}(n_1+1)\,p^{n_1}\,\rm {d}p}$$ (5.51)

$$= (n_1+n_2+1)\,p^{n_1+n_2}\, ,$$ (5.52)

getting the same result.