Next: Distributions (a concise reminder)
Up: Choice of the initial
Previous: General criteria
Contents
The first and most famous criterion for choosing
initial probabilities is the simple
Principle of Insufficient Reason
(or Indifference Principle): If there is no reason
to prefer one hypothesis over alternatives, simply attribute
the same probability to all of them. This
was stated as a principle
by Laplace^{3.13}
in contrast to
Leibnitz' famous Principle of Sufficient Reason, which, in simple
words, states that ``nothing happens without a reason".
The indifference principle applied to coin and die tossing,
to card games or to other simple and symmetric
problems, yields to the wellknown rule of probability
evaluation that we have called combinatorial.
Since it is impossible not to agree with this point of
view, in the cases for which one judges that it does apply,
the combinatorial ``definition'' of probability is
recovered in the Bayesian approach if the
word ``definition'' is simply replaced by ``evaluation rule''.
We have in fact already used this reasoning
in previous examples.
A modern and more sophisticated version of the Indifference Principle
is the Maximum Entropy Principle. The information entropy
function of
mutually exclusive events, to each of which a probability
is assigned, is defined as[40]

(3.25) 
with a positive constant. The principle states that
``in making inferences on the basis of partial
information we must use that probability distribution which
has the maximum entropy subject to whatever is known[41]''.
Note that, in this case, ``entropy'' is synonymous with
``uncertainty''[41].
One can show that, in the case of absolute
ignorance about the events , the maximization of the
information uncertainty, with the constraint that
,
yields the classical
(any other result would have been worrying ).
Although this principle is sometimes used in combination
with the Bayes formula for inferences
(also applied to measurement uncertainty, see Ref. [23]), it will not be
used for applications in these notes.
Next: Distributions (a concise reminder)
Up: Choice of the initial
Previous: General criteria
Contents
Giulio D'Agostini
20030515