Fake claims of discoveries are mainly caused by
statistical prescriptions that do not follow
probabilistic reasoning, meant as mathematics
of beliefs, as it was conceived as a whole by
Laplace and that nowadays is known under
the appellative `Bayesian'. As a consequence
- the concept of probability of causes is refused;
- the role of Bayes' theorem to update beliefs
is rejected, and hence
- the role of prior knowledge is not explicitly recognized;
- the myth has been created that a single hypothesis can be `tested'
without taking explicitly into account alternative(s);
- the intuitive concept of `probabilities of causes'
has been surrogated by ad hoc hypothesis test
prescriptions,
- whose choice and use are rather arbitrary;
- whose results are routinely misinterpreted.
Unfortunately, this wobbly construction faces against the human
predisposition to think naturally in terms of degrees of belief
about anything we are in condition of uncertainty, including
the several causes that might have produced the observed effects.
The result of this mismatch is that
- probabilities of the effects given the causes are
confused with the probabilities of the causes given the effects;
- even worse, p-values are used as if they were
the probability that the hypothesis under test is true.
In addition, the pretension that `priors are not scientific and should
not enter the game' (``the data should speak by themselves'')
avoids that sound scientific priors mitigate the deleterious effects
of misunderstood p-values.
But, fortunately, being the natural intuition of physicists
rather `Bayesian'[20], after all it is more
a question of rough scientific communication
than of rough science.
In fact, even the initial
excitement of someone who takes a bit too seriously
claims that the rest of the physics community
classifies immediately as `fake' - priors! -
is harmless, if the discussions remain in the community.
And the debates are often even profitable, because they
offer an opportunity to check how new possible phenomena
and new explanations could fit into the present
network of beliefs
based on all previous experimental observations. This
is for example what has recently happened with the exchange
of ideas that has followed
the Opera result on neutrino speed, from which
most of us have learned something.
As far as the communication of claims to non experts, that
include also physicists of other branches, or even of a close
sub-branch, my recommendation is of making use,
at least qualitatively, of the Bayesian odd update, i.e.
- state how much the experimental data push towards
either possibility (that is the Bayes factor,
which has nothing to do with p-values);
- state also how believable are the two hypotheses
independently of the data in object.
I am pretty sure most people can make a good use of these
pieces of information. Moreover, my recommendation to
journalist and opinion makers (including bloggers and similar)
is that, in the case of doubt:
- don't accept answers in terms of p-values,
unless you are sure you understand them well and
you feel capable to explain their correct meaning
to the general public
without they become somehow probabilities of the hypotheses
to be compared (good luck!);
- refuse as well `confidence levels',
`95% confidence exclusion curves' and similar;
- ask straight the direct questions:
- How probable it is? (Possibly informing - threatening! -
him/her in advance that his/her answer will be
reported as ``Dr X.Y. considers
it such and such percent
probable''.)
- How much do you believe? (Same as the previous one.)
- How much would you believe in either hypothesis
if you did not have this data? (The answer
allows you to estimate the priors odds.)
- How much would you believe in either hypothesis
given these data, if you considered the two hypotheses
initially equally probable? (The answer
allows you to evaluate the Bayes factor.)
- How much would you bet in favor of your claim?
(And if you realize there are the conditions described
in section 5.3 and figure
2, don't miss the opportunity
to gain some money!)
To end, I would like to congratulate all people
working at LHC on the amazing high quality
work done in these years and on having been able to report
these convincing hints on the Higgs boson in a record time (I had never betted
in favor of this possibility in 2011 even six months ago!
But now the real exciting bet is what next?).
Giulio D'Agostini
2012-01-02