It is surprising that many physicists have been slow to adopt these `new' ideas. There have been notable exceptions, of course, many of whom have contributed to the abovementioned Maximum Entropy workshops. One reason to be surprised is because numerous great physicists and mathematicians have played important roles in developing probability theory. These `new' ideas actually originated long ago with Bernoulli, Laplace, and Gauss, just to mention a few who contributed significantly to the development of physics, as well as to Bayesian thinking. So, while modern statisticians and mathematicians are developing powerful methods to apply to Bayesian analysis, most physicists, in their use and reasoning in statistics still rely on 20 century `frequentist prescriptions' (D'Agostini 1999a, 2000).
We hope that this report will help fill this gap by reviewing the advantages of using the Bayesian approach to address physics problems. We will emphasize more the intuitive and practical aspects than the theoretical ones. We will not try to cover all possible applications of Bayesian analysis in physics, but mainly concentrate on some basic applications that illustrate clearly the power of the method and how naturally it meshes with physicists' approach to their science.
The vocabulary, expressions, and examples have been chosen with the intent to correspond, as closely as possible, to the education that physicists receive in statistics, instead of a more rigorous approach that formal Bayesian statisticians might prefer. For example, we avoid many important theoretical concepts, like exchangeability, and do not attempt to prove the basic rules of probability. When we talk about `random variables,' we will in fact mean `uncertain variables,' and instead of referring to the frequentist concept of `randomness' à la von Mises (1957). This distinction will be clarified later.
In the past, presentations on Bayesian probability theory often start with criticisms of `conventional,' that is, frequentist ideas, methods, and results. We shall keep criticisms and detailed comparisons of the results of different methods to a minimum. Readers interested in a critical review of conventional frequentist statistics will find a large literature, because most introductory books or reports on Bayesian analysis contain enough material on this matter. See (Gelman et al 1995, Sivia 1997, D'Agostini 1999c, Jaynes 1998, Loredo 1990) and the references therein. Eloquent `defenses of the Bayesian choice' can be found in (Howson and Urbach 1993, Robert 2001).
Some readers may wish to have references to unbiased comparisons of frequentist to Bayesian ideas and methods. To our knowledge, no such reports exist. Those who claim to be impartial are often frequentists who take some Bayesian results as if they were frequentist `prescriptions,' not caring whether all underlying hypotheses apply. For two prominent papers of this kind, see the articles by Efron (1986a) [with follow up discussions by Lindley (1989), Zellner (1986), and Efron (1986b)] and Cousins (1995). A recent, pragmatic comparisons of frequentist and Bayesian confidence limits can be found in (Zech 2002).
Despite its lack of wide-spread use in physics, and its complete absence in physics courses (D'Agostini 1999a), Bayesian data analysis is increasingly being employed in many areas of physics, for example, in astronomy (Gregory and Loredo 1992, 1996, Gregory 1999, Babu and Feigelson 1992, 1997, Bontekoe et al 1994), in geophysics (Glimm and Sharp 1999), in high-energy physics (D'Agostini and Degrassi 1999, Ciuchini et al 2001), in image reconstruction (Hanson 1993), in microscopy (Higdon and Yamamoto 2001), in quantum Monte Carlo (Gubernatis et al 1991), and in spectroscopy (Skilling 1992, Fischer et al 1997, 1998, 2000), just to mention a few articles written in the last decade. Other examples will be cited throughout the paper.