Uncritical or wishful use of p-values can be dangerous,
not to speak of unscrupulous p-hacking. While years ago
these criticisms were raised by a minority of thorny Bayesians,
now the effect on the results in
several fields of science and technology
is felt as a primary issue.32
The statement of the American
Statistical Association is certainly commendable in
addressing the issue, but it is
in my opinion unsatisfactory not admitting
that the question is inherent to all statistical methods
that refuse the very idea of probability of hypotheses, or
of ``probability of causes'', i.e.
what Poincaré used to call
``the essential problem of the experimental method.''
While I had experienced several times in the past,
including this winter,
claims of possible breaking discoveries in Particle Physics simply
due to misinterpretations of p-values, for the first time
I have realized of a case in which judgements based on p-values
strongly reduce the `significance' of important results.
This happens with the gravitational wave events reported
this year by the LIGO-Virgo collaboration, and in particular
with the October 12 events timidly reported as
a LIGO-Virgo Trigger (`Cinderella'), because of its 1.7 sigmas, in spite
of the huge Bayes factor of about
, that should instead
convince any hesitating physicist about its nature
of a gravitational wave radiated by a Binary Black Hole merger,
especially in the light of the other, more solid two events
(`the two sisters').33I hope than that LVT151012 will be upgraded to
GW151012 and that in future searches the Bayes factor
will become the principal figure of merit to rank
gravitational wave candidates.
I finally conclude with some questions asked at the end of
talk on which this paper is based.
- Which Bayes factor would characterize the 750GeV excess?
The result depends on the model to explain the
excess34
and an answer came the week after MaxEnt 2016 by
Andrew Fowlie[66]. For the model considered
he got a BF around 10, the exact value being irrelevant:
a weak indication, but nothing striking to force
sceptics to change substantially their
opinion.35
- Could have CDF at Fermilab claimed
to have observed the Higgs boson
if they had done a Bayesian analysis?
I am quite positive they could have it, also because
the prior on the possible values of the Higgs mass
was not so vague and well matching the
value found later, and therefore the Bayes Factor
would have been rather high (and the prior probability
of a possible manifestation of the boson in the
final state was high too).
Acknowledgements
This work was partially supported by a grant from Simons Foundation,
which allowed me a stimulating working environment during
my visit at the Isaac Newton Institute
of Cambridge, UK. The understanding and/or presentation
of several things
of this paper has benefitted
of the interactions with Pia Astone, Ariel Caticha,
Kyle Cranmer, Walter Del Pozzo, Norman Fenton,
Enrico Franco, Gianluca Gemme, Stefano Giagu,
Massimo Giovannini, Keith Inman,
Gianluca Lamanna, Paola Leaci,
Marco Nardecchia,
Aleandro Nisati,
and Cristiano Palomba. I am particularly indebded to Allen Caldwell,
Alvaro de Rujula and John Skilling for many discussions
on physics, probability, epistemology and sociology of
scientific communities, as well for valuable comments
on the manuscript, which has also benefitted of an accurate reading
by Christian Durante and
Dino Esposito.
Giulio D'Agostini
2016-09-06