Inferring $p$ in absence of background

The solution of Eq.(4) depends, at least in principle, on the assumption on the prior $f_\circ(x)$. Taking a flat prior between 0 and 1, that models our indifference on the possible values of $p$ before we take into account the result of the experiment in which $x$ successes were observed in $n$ trials, we get (see e.g. [2]):

$$f(p\,\vert\,x,n,{\cal B}) = \frac{(n+1)!}{x!\,(n-x)!}\,p^x\,(1-p)^{n-x}\,,$$ (12)

some examples of which are shown in Fig. 1.

Figure: Probability density function of the binomial parameter $p$, having observed $x$ successes in $n$ trials.[2]

Expected value, mode (the value of $p$ for which $f(p)$ has the maximum) and variance of this distribution are:

$$\mbox{E}(p) = \frac{x+1}{n+2}$$ (13)

$$\mbox{mode}(p)= p_m = x/n$$ (14)

$$\sigma^2(p)=\mbox{Var}(p) = \frac{(x+1)(n-x+1)}{(n+3)(n+2)^2}$$ (15)

$$= \mbox{E}(p)\,\left(1 - \mbox{E}(p)\right)\,\frac{1}{n+3}\,.$$ (16)

Eq. (13) is known as ``recursive Laplace formula'', or ``Laplace's rule of succession''. Not that there is no magic if the formula gives a sensible result even for the extreme cases $x=0$ and $x=n$ for all values of $n$ (even if $n=0$!). It is just a consequence of the prior: in absence of new information, we get out what we put in!

From Fig. 1 we can see that for large numbers (and with $x$ far from 0 and from $n$) $f(p)$ tends to a Gaussian. This is just the reflex of the limit to Gaussian of the binomial. In this large numbers limit $\mbox{E}(p) \approx p_m =x/n$ and $\sigma(p) \approx \sqrt{x/n\,(1-x/n)/n}$.