Biased Bayesian estimators and Monte Carlo checks of

Bayesian procedures

Let us consider the case of Poisson and binomial distributed observations, exactly as they have been treated in Sections and , i.e. assuming a uniform prior. Using the typical notation of frequentistic analysis, let us indicate with the parameter to be inferred, with its estimator.

**Poisson:**-
; indicates the possible observation
and
is the estimator in the light of :

E E E (8.3)

The estimator is biased, but consistent (the bias become negligible when is large). **Binomial:**-
; after trials one may observe
favourable results, and the estimator of is then

E E E (8.4)

In this case as well the estimator is biased, but consistent.

- the initial intent is to reconstruct at best the parameter, i.e. the true value of the physical quantity identified with it;
- the freedom from bias requires only that
the expected value of the estimator
should equal the value of the
parameter,
for a given value of the parameter,

E e.g. E i.e. (8.5)

There is another important and subtle point related to this problem, namely that of the Monte Carlo check of Bayesian methods. Let us consider the case depicted in Fig. and imagine making a simulation, choosing the value , generating many (e.g. 10 000) events, and considering three different analyses:

- a maximum likelihood analysis;
- a Bayesian analysis, using a flat distribution for ;
- a Bayesian analysis, using a distribution of `of the kind' of Fig. , assuming that we have a good idea of the kind of physics we are doing.

Now, let us assume we have observed a value of , for example . Which analysis would you use to infer the value of ? Considering only the results of the Monte Carlo simulation it seems obvious that one should choose one of the first two, but certainly not the third!

This way of thinking is wrong, but unfortunately
it is often used by practitioners who have no time to understand
what is behind Bayesian reasoning, who perform some Monte Carlo tests,
and decide that the Bayesian theorem does not
work!^{8.12} The solution
to this apparent paradox is simple.
If __you__
believe that is distributed like
of Fig. , then you should use this
distribution in the analysis and also
in the generator. Making a simulation
based only on a single true value, or on a set of
points with equal weight, is equivalent to assuming
a flat distribution for and, therefore,
it is not surprising
that the most grounded Bayesian analysis is that
which performs worst
in the simple-minded frequentistic checks.
It is also worth remembering that priors are not
just mathematical objects to be plugged into Bayes' theorem,
but must reflect prior knowledge. Any inconsistent
use of them leads to paradoxical results.