Bayesian Reasoning in High Energy Physics
- Principles and Applications -

Giulio D'Agostini
Dip. di Fisica Università ``La Sapienza'' and INFN, Roma, Italy

An improved and largely extended version of this report has been published in 2003 as

Bayesian reasoning in data analysis: A critical introduction

• Contents
  • .
    • Introduction
  
  
• Subjective probability in Physics? Scientific reasoning in conditions of uncertainty
  • Uncertainty in physics and the usual methods of handling it
    • Uncertainty in physics
    • True value, error and uncertainty
    • Sources of measurement uncertainty
    • Usual handling of measurement uncertainties
    • Probability of observables versus probability of true values
    • Probability of the causes
    • Unsuitability of confidence intervals
    • Misunderstandings caused by the standard paradigm of hypothesis tests
    • Statistical significance versus probability of hypotheses
  • A probabilistic theory of measurement uncertainty
    • Where to restart from?
    • Concepts of probability
    • Subjective probability
    • Learning from observations: the `problem of induction'
    • Beyond Popper's falsification scheme
    • From the probability of the effects to the probability of the causes
    • Bayes' theorem for uncertain quantities: derivation from a physicist's point of view
    • Afraid of `prejudices'? Inevitability of principle and frequent practical irrelevance of the priors
    • Recovering standard methods and short-cuts to Bayesian reasoning
    • Evaluation of uncertainty: general scheme
      • Direct measurement in the absence of systematic errors
      • Indirect measurements
      • Systematic errors
      • Approximate methods
  
  
• ``Bayesian primer''
  - slightly reviewed version of the 1995 DESY/Rome report -
  • Subjective probability and Bayes' theorem
    • Orig inal abstract of the ``primer''
    • Introduction to the ``primer''
    • Probability
      • What is probability?
      • Subjective definition of probability
      • Rules of probability
      • Subjective probability and ``objective'' description of the physical world
    • Conditional probability and Bayes' theorem
      • Dependence of the probability on the state of information
      • Conditional probability
      • Bayes' theorem
      • Conventional use of Bayes' theorem
      • Bayesian statistics: learning by experience
    • Hypothesis test (discrete case)
    • Choice of the initial probabilities (discrete case)
      • General criteria
      • Insufficient reason and maximum entropy
  • Distributions (a concise reminder)
    • Random variables
      • Discrete variables
      • Continuous variables: probability and density function
      • Distribution of several random variables
    • Central limit theorem
      • Terms and role
      • Distribution of a sample average
      • Normal approximation of the binomial and of the Poisson distribution
      • Normal distribution of measurement errors
      • Caution
  • Bayesian inference applied to measurements
    • Measurement errors and measurement uncertainty
    • Statistical inference
      • Bayesian inference
      • Bayesian inference and maximum likelihood
      • The dog, the hunter and the biased Bayesian estimators
    • Choice of the initial probability density function
      • Difference with respect to the discrete case
      • Bertrand paradox and angels' sex
    • Normally distributed observables
      • Final distribution, prevision and credibility intervals of the true value
      • Combination of several measurements
      • Measurements close to the edge of the physical region
    • Counting experiments
      • Binomially distributed observables
      • Poisson distributed quantities
    • Uncertainty due to systematic errors of unknown size
      • Example: uncertainty of the instrument scale offset
      • Correction for known systematic errors
      • Measuring two quantities with the same instrument having an uncertainty of the scale offset
      • Indirect calibration
      • Counting measurements in the presence of background
  • Bypassing the Bayes' theorem for routine applications
    • Approximate methods
      • Linearization
      • BIPM and ISO recommendations
      • Evaluation of type B uncertainties
      • Examples of type B uncertainties
      • Caveat concerning the blind use of approximate methods
    • Indirect measurements
    • Covariance matrix of experimental results
      • Building the covariance matrix of experimental data
      • Use and misuse of the covariance matrix to fit correlated data
  • Bayesian unfolding
    • Problem and typical solutions
    • Bayes' theorem stated in terms of causes and effects
    • Unfolding an experimental distribution
  
  
• Other comments, examples and applications
  • Appendix on probability and inference
    • Unifying role of subjective approach
    • Frequentists and combinatorial evaluation of probability
    • Interpretation of conditional probability
    • Are the beliefs in contradiction to the perceived objectivity of physics?
    • Biased Bayesian estimators and Monte Carlo checks of Bayesian procedures
    • Frequentistic coverage
    • Bayesian networks
    • Why do frequentistic hypothesis tests `often work'?
    • Frequentists and Bayesian `sects'
      • Bayesian versus frequentistic methods
      • Orthodox teacher versus sharp student - a dialogue by Gabor
      • Subjective or objective Bayesian theory?
      • Bayes' theorem is not all
    • Solution to some problems
      • AIDS test
      • Gold/silver ring problem
  • Further HEP applications
    • Poisson model: dependence on priors, combination of results and systematic effects
      • Dependence on priors
      • Combination of results from similar experiments
      • Combination of results: general case
      • Including systematic effects
      • Is there a signal?
      • Signal and background: a Mathematica example
    • Unbiased results
      • Uniform prior and fictitious quantities
    • Constraining the mass of a hypothetical new particle: analysis strategy on a toy model
      • The rules of the game
      • Analysis of experiment $A$
      • Naïve procedure
      • Correct procedure
      • Interpretation of the results
      • Outside the sensitivity region
      • Including other experiments
  
  
• .
  • Conclusions
    • About subjective probability and Bayesian inference
    • Conservative or realistic uncertainty evaluation?
    • Assessment of uncertainty is not a mathematical game
    • Acknowledgements
    • Bibliographic note
  
  
• Bibliography
• About this document ...