[ Bayesian inference and its application to physics measurements ]

(For most of material in Italian see here)

"Bayesian reasoning in data analysis - A critical introduction", World Scientific Publishing 2003

"Statistics books must take seriously the need to teach the foundations of statistical reasoning from the beginning...
D'Agostini's new book does it admirably, building an edifice od Bayesian statistical reasoning in the physical sciences on solid foundations.

According to the author, the audience for this book is practicing physicists and engineers who need to evaluate uncertainty. To that list, I would like to add upper-level and graduate students in physics and engineering, statisticians and statistics graduate students who collaborate with physicists and engineers, mathematicians who teach statistics courses, and all Bayesian statisticians" (J. Am. Stat. Ass.)

• "Judgement Leanings, a graphical way to hopefully improve the perception and the communication of the `weights of evidence' and of the 'intensities of beliefs", poster at PESFoG2012: Slides (110k).

• “Scoperte annunciate a colpi di 'sigma'”, in Italian: vedi qui per slides, video, info sull'evento e link collegati.

• "Probably a discovery: Bad mathematics means rough scientific communication", arXiv:1112.3620v2: pdf (383k) [html version, containing also additional related links].

• "Reti Bayesiane: da modelli di conoscenza a strumenti inferenziali e decisionali" (con Serena Cenatiempo e Aldo Vannelli), Notiziario Tecnico di Telecom Italia, 2010, Numero 3, pp 16-25 (copia locale, pdf, 836k).

• "Improved iterative Bayesian unfolding", arXiv:1010.0632v1: ps.gz (384k), pdf (321k).
  • Old version
  • Slides presented at the Alliance Workshop on Unfolding and Data Correction, Hamburg, Germany, 27-28 May 2010.
  • Slides presented at the Helmholtz School “Advanced Topics in Statistics”, Göttingen, 17-20 October 2010 (with references to several topics discussed there, most of them not strictly related to unfolding).
  • R code: unf2_R.tgz (12k).
  • C, C++, Root versions? They are warmly welcome.

• "On the Peirce's balancing reasons rule failure in his "large bag of beans" example", arXiv:1003.3659v1: ps.gz (113k), pdf (148k).

• "A defense of Columbo (and of the use of Bayesian inference in forensics): A multilevel introduction to probabilistic reasoning", arXiv:1003.2086v2: ps.gz (3.4M), pdf (771k) and html (the automatic translation from latex is quite poor).
  • Hugin Bayesian network of the toy model of the paper: box_testimony.oobn.
  • Related links:
    • "A formula for justice", by A. Saini The Guardian, 2 October 2011
    • Bayes and the Law

• "L'inferenza probabilistica: ruolo nelle scienze sperimentali e suggerimenti per il suo insegnamento" (Probabilistic inference: role in the experimental sciences and suggestion for its teaching) invited contribution to the Dossier Statistica, TRECCANI Scuola, February 2010.

• "On the so-called Boy or Girl Paradox", arXiv:1001.0708v1: ps.gz (93K) and pdf (150K).

• Sleeping Beauty problem:
  • play the Sleeping Beauty game;
  • note in preparation:

• "About the proof of the so called exact classical confidence intervals. Where is the trick?", physics/0605140v2: ps.gz (73K) and pdf (125K).

• "Dalle osservazioni alle ipotesi scientifiche" (From the observations to the scientific hypotheses), in Italian: invited contribution to TRECCANI Scuola, Aprile 2006.
  Versione locale.

• "Fits, and especially linear fits, with errors on both axes, extra variance of the data points and other complications", physics/0511182: ps.gz (191K), pdf (259K) and HTML.

• The Fermi's Bayes Theorem

• “C'è statistica e statistica”, risposta ad una domanda su “statistica gaussiana e statistica bayesiana” rivolta a Scienza per tutti dei Laboratori Nazionali di Frascati, pubblicata su Scaffali, 30 marzo 2005 (pdf, 45kB; copia locale).

• "Telling the truth with statistics". Lectures for the CERN Academic Training, 21-25 February 2005.
  • Pdf files of transparancies: 1, 2, 3, 4, 5 (to be viewed in fullscren with acroread).
  • Transparencies (as well as videos) also available at the CERN agenda server.
  • The ultimate confidence intervals calculator [see lecture 2, p.24 (slide 95)].
  (The title of lectures, commented in the first lecture, was proposed by the organisers.)

• "From Observations to Hypotheses: Probabilistic Reasoning Versus Falsificationism and its Statistical Variations". Invited talk at the 2004 Vulcano Workshop on Frontier Objects in Astrophysics and Particle Physics, Vulcano (Italy) May 24-29, 2004, physics/0412148v2: ps.gz (111k) or pdf (259k) file; HTML version.

• "Inferring the success parameter p of a binomial model from small samples affected by background", physics/0412069: ps.gz (1.1M) or pdf (403k) file; HTML version.

• "Asymmetric uncertainties: sources, treatment and possible dangers", physics/0403086: ps.gz (357k) or pdf (236k) file; HTML version.
  

• "Bayesian inference in processing experimental data: principles and basic applications", invited paper for Reports on Progress in Physics [66 (2003)1383]
  Preprint: physics/0304102: ps.gz (281k), pdf (473k), dvju (304k); HTML version.
  

  The same issue of RPP contains also a review by Volker Dose on "Bayesian inference in physics: case studies": 66 (2003)1421 (many related publications of Dose and collaborators on data analysis can be found in the IPP Bayesian Data Analysis Group web page).

• "Bayesian model comparison applied to the Explorer-Nautilus 2001 coincidence data", with Pia Astone and Sabrina D'Antonio, gr-qc/0304096: ps.gz (164k), pdf (160k) and html (160k) versions. Published in Class. Quant. Grav. 20 (2003) 769.
  Note added (December 2004): as stated in my contribution to Vulcano04 (footnote 9), this paper should be taken more for its methodological contents than for the physical outcome.

• "Search for correlation between GRB's detected by BeppoSAX and gravitational wave detectors EXPLORER and NAUTILUS", astro-ph/0206431 (with Explorer/Nautilus and BeppoSAX Collaborations). Published in Phys Rev D66 (2002) 102002.

• "Minimum bias legacy of search results", Seventh Topical Seminar on "The legacy of LEP and SLC", Siena, Italy, 8-11 October 2001. hep-ph/0201070: ps.gz (35k) or PDF (143k) file. Nucl Phys Proc Suppl 109 (2002) 148-152.

• "Inference of rho and eta of the CKM matrix - A simplified, intuitive approach" (based on a series of lectures given at the VI LNF Spring School "Bruno Touscheck", Frascati, Italy, May 2001, hep-ph/0107067): ps.gz (1.1MB) or PDF (1.6MB) file.
  html version

• "2000 CKM-Triangle Analysis: A critical review with updated experimental inputs and theoretical parameters" (with M. Ciuchini, E. Franco, V. Lubicz, G. Martinelli, F. Parodi, P. Roudeau A. Stocchi) hep-ph/0012308: ps.gz (486k) or PDF (1.2MB) file. Published in JHEP 0107 (2001) 013.
  I had some fun with referees also with this paper:
  • Referee report: ps file and pdf file
  • Our reply
  • Finally, the paper was accepted "with the benefit of the doubt": ... speachless....
  For more info and plots about the Unitary Triangle analysis, see www.UTfit.org.

• ``Role and meaning of subjective probability: some comments on common misconceptions'', talk at the XXth International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, July 8-13,2000, Gif sur Yvette (Paris), France: (Maxent00): postscript file or PDF file (also at arXiv: physics/0010064) AIP Conference Proceedings (Melville) Vol. 568 (2001) 23-30.

• ``Teaching Bayesian statistics in the scientific curricula'', The ISBA Newletter (later renamed 'bulletin'), Vol. 7, No. 1 (March 2000) pp. 18-19, (local txt file).

• ``Uncertainties due to imperfect knowledge of systematic effects: general considerations and approximate formulae'' (with Mirko Raso), CERN-EP/2000-026, February 2000, hep-ex/0002056: postscript file or PDF file.
  A reduced version of this paper has become a part of Chapter 12 of Bayesian reasoning in data analysis.

• Workshop on `Confidence Limits' at CERN, Jan. 17-18, 2000: CERN document server.
  • Paper based on the invited talk Confidence limits: what is the problem? Is there the solution? (eprint hep-ex/0002055): postscript file or PDF file (Mathematica file containing the Newton problem and the box example, see also the errata below!).
  • Contributed paper with G. Degrassi: Constraining the Higgs boson mass through the combination of direct search and precision measurement result (an update of our 1999 EPJ paper): postscript file or PDF file (also hep-ph/0001269).
  • Short contribution during the pannel and discussion session: Quantum mechanics and probability (with comments on confidence intervals): transparencies (see also organizers' transcript of panel discussion).
  • "Conclusions" of the Workshop.
  • Proceedings published as CERN Report 2000-005.
  • 20/08/2001: errata about the numbers of the "original" problem to Newton.

• ``Sceptical combination of experimental results: General considerations and application to epsilon-prime/epsilon", CERN-EP/99-139, October 1999, hep-ex/9910036: postscript file or PDF file. This paper will remain unpublished. People interested in sociology of science might want to give a look at 1st referee report, my replay, 2nd referee report, 2nd replay and final verdict.
  This paper, with little changes, has become Chapter 11 of Bayesian reasoning in data analysis. Please refer to that book for citations.

• ``Inferring the intensity of Poisson processes at limit of detector sensitivity (with a case study on gravitational wave burst search)" (with Pia Astone, special acknowledgements to Maddalena...), CERN-EP/99-126, August 1999, hep-ex/9909047: postscript file or PDF file.
  The substance of this paper has become a part of Chapter 13 of Bayesian reasoning in data analysis.

• "Teaching statistics in the physics curriculum. Unifying and clarifying role of subjective probability", invited contribution to the special theme issue of the American Journal of Physics on Thermal and Statistical Physics (H. Gould and J. Tobochnik eds.):
  • Preprint Roma1-1108 (July 1999) and LANL physics/9908014 (August 1999): postscript file or PDF file.
  • Published in AJP 67, issue 12 (1999) 1260-1268: html version (for abstract, pdf and ps versions see here).

• ``Bayesian reasoning in high energy physics. Principles and applications'',
  CERN Yellow Report 99-03, July 1999 (vi + 175 pages).
  • html version (latex2html translation not really perfect...)
  • ps/pdf version (by parts):
    • Introduction + contents (8 pages): ps file, PDF file;
    • Part I - Subjective probability in Physics? Scientific reasoning in conditions of uncertainty (36 pages): ps file, PDF file;
    • Part II - Updated version of "Bayesian primer" (78 pages): ps file, PDF file;
    • Part III - Other comments, examples and applications (44 pages): ps file, PDF file;
    • Concluding matter + bibliography (15 pages): ps file, PDF file;
  (Also available at CERN Document Server).
  For FAQ's and clarifications see here (all comments are welcome!).
  A reviewed and extended version of the report has been published in 2003 by World Scientific Publishing.

• ``Overcoming prior anxiety'', invited contribution to the monographic issue of the Revista de la Real Academia de Ciencias on Bayesian Methods in the Sciences (J.M. Bernardo ed.); postscript file or PDF file (also at Los Alamos xxx archive: physics/9906048).

• Tutorial at Los Alamos Laboratory (March 1-3, 1999): for writeup of matter concerning lessons see report based on CERN lectures.
  The "six boxes" example, of which the viewgraphs are available in pdf format at LNAL, has been included in the AJP paper ("Teaching statistics ..."), see above in this list.

• ``Constraints on the Higgs Boson Mass from Direct Searches and Precision Measurements'' (with G. Degrassi), DFPD-99/TH/02, February 1999, hep-ph/9902226, published in Eur. Phys. J. C10 (1999) 663-675: local postscript file or PDF file of preprint.
  You might be interested in what a `first rate' frequentist referee would think about the paper: see referee report.
  Updated analysis as contribution to the Workshop on `Confidence limits' (see above): hep-ph/0001269.

• "Observation of a tossed coin remaining vertical" (15 Nov 1998), plus other oddities.

• "Bayesian Reasoning versus Conventional Statistics in High Energy Physics", invited talk at the XVIII International Workshop on Maximum Entropy and Bayesian Methods (Maxent98): postscript file or PDF file (also at Los Alamos xxx archive: physics/9811046). html version at Caltech

• "Tutorial on measurement uncertainty" at EC Summer School: Bayesian Signal Processing, Cambridge, July 1998. (No proceedings; note in preparation with some original material only presented there).

• "Jeffreys Priors versus Experienced Physicist Priors - Arguments against Objective Bayesian Theory",
  contributed paper to "Valencia 6" (1998): postscript file or PDF file (also at Los Alamos xxx archive: physics/9811045).

• ``Bayesian Reasoning in High Energy Physics: Principles and Applications'',
  lecture notes of academic training at CERN (25-29 May 1998)- videos of lectures also available at CERN Library.
  ** Final version **: CERN Yellow Report 99-03 (see above).

• ``Measurements errors and measurement uncertainty - critical review and proposals for teaching'', Internal Report n. 1094, Phys. Dept., May 1998: postscript file or PDF file, in italian (86 pages):
  this is my manifesto concerning the use of subjective probability for the evaluation of uncertainty in measurement as well as for a teaching method of physics laboratory based on the Bayesian motto "Learning by Experience".

• ``A theory of measurement uncertainty based on conditional probability''
  ROME1-1079-1996, Nov 1996, physics/9611016 (presented at JSM 1996 at Chicago, IL).

• ``Probability and Measurement Uncertainty in Physics - a Bayesian Primer''
  DESY-95-242, Roma1 N. 1070, hep-ph/9512295 (original postscript file also in DESY).
  An updated version is contained in the CERN Yellow Report 99-03 (see above).
  A rivised version is now included in Bayesian Reasoning

  Several ideas of this paper have been applied in the following notes by Mirko Raso (including Bayesian unfolding and treatment of correlations due to systematics):

  • ZEUS-96-132, "Measurement of F₂ep from the 1995 shifted vertex data": ps and pdf file.
  • ZEUS-98-074, "An update of the F₂ep using the 1995 shifted vertex data": ps and pdf file; html version.
  • ZEUS-98-075, "A measurement of the sigma_ep for Q² < 0.3 GeV², as a by product of the F₂ep measurement": ps and pdf file; html version.

• Bayesian unfolding:
  • Paper: Nucl. Instr. and Meth. in Phys. Res. A362 (1995) 487
    • scanned version of DESY preprint [local files: pdf(576k) and djvu (410k)];
    • citations from SPIRES data base
  • distribution file with FORTRAN code (62k);
  • Comments added: ps file (60k) and pdf file (65k).
  • Speeding up the code: comments by Alex Grossheim (03/06/03).
  • A C++ implementation of the program, that make use of ROOT libraries, is available at Emanuele Di Marco unfolding page.
  • For an implementation in Root, see e.g. RooUnfold (there migh be others).
  For examples of analysis in which the Bayesian unfolding has been used, see Mirko Raso collaboration notes on low x Deep Inelastic Scattering at HERA. (see also SLAC SPIRES Database for some papers that used the program).
  →Improved version

• ZEUS Note (1995) advertising ISO Guide: postscript file;

• ``On the use of the covariance matrix to fit correlated data'',
  DESY-93-175, December 1993,
  Nucl. Instr. and Meth. in Phys. Res. A346 (1994) 306.
  (scanned version of DESY preprint at KEK);

• ``Quark-gluon jet separation in the photoproduction region with a neural network algorithm'' (with G. Barbagli and D. Monaldi), Roma1-992, Feb. 1992; Proc. of Workshop on Physics at HERA, Hamburg, Germany, October 29-30, 1991;

• "Limits on the electron compositeness from the Bhabha scattering at PEP and PETRA".
  25th Rencontre de Moriond on Electroweak Interactions and Unified Theories, Les Arcs, France, Mar 4-11, 1990. (scanned version of Rome preprintat KEK);

• "Determination of alpha_s and the Z0 mass from measurements of the total hadronic cross section in e+e- annihilation" (with W. De Boer and G. Grindhammer),
  Phys. Lett. B229 (1989) 160. (scanned version of DESY preprint at KEK);

• "Determination of alpha_s and sin^2theta(W) from the R measurements at PEP and PETRA", 22th Rencontre de Moriond, Les Arcs, France, Mar 15-21 1997, Editions Frontières, pp. 325-336.

• "Determination of alpha_s and sin^2theta(W) from measurements of the total hadronic cross section in e+e- annihilation" (with CELLO Collaboration),
  Phys. Lett. 183B (1987) 400. (scanned version of DESY preprint at KEK);

Related web sites

Back to G.D'Agostini Home Page