Predictive distributions

A related problem is to `infer' what an experiment will observe given our best knowledge of the underlying theory and its parameters. Infer is within quote marks because the term is usually used for model and parameters, rather than for observations. In this case people prefer to speak about prediction (or prevision). But we recall that in the Bayesian reasoning there is conceptual symmetry between the uncertain quantities which enter the problem. Probability density functions describing not yet observed event are referred to as predictive distributions. There is a conceptual difference with the likelihood, which also gives a probability of observation, but under different hypotheses, as the following example clarifies.

Given $\mu$ and $\sigma$, and assuming a Gaussian model, our uncertainty about a `future' $d_f$ is described by the Gaussian pdf Eq. (25) with $d=d_f$. But this holds only under that particular hypothesis for $\mu$ and $\sigma$, while, in general, we are also uncertain about these values too. Applying the decomposition formula (Tab. 1) we get:

$$p(d_f\,\vert\,I) = \int p(d_f\,\vert\,\mu,\sigma,I)\,p(\mu,\sigma\,\vert\,I) \,\mbox{d}\mu\,\mbox{d}\sigma$$ (58)

Again, the integral might be technically difficult, but the solution is conceptually simple. Note that, though the decomposition formula is a general result of probability theory, it can be applied to this problem only in the subjective approach.

An analytically easy, insightful case is that of experiments with well-known $\sigma$'s. Given a past observation $d_p$ and a vague prior, $p(\mu\,\vert\,d_p,I)$ is Gaussian around $d_p$ with variance $\sigma^2_p$ [note that, with respect to $p(\mu,\sigma\,\vert\,I)$ of Eq.(58), it has been made explicit that $p(\mu)$ depend on $d_p$]. $p(d_f\,\vert\,\mu)$ is Gaussian around $\mu$ with variance $\sigma^2_f$. We get finally

$$p(d_f\,\vert\,d_p,I) = \int p(d_f\,\vert\,\mu,I)\,p(\mu\,\vert\,d_p,I)\,\mbox{d}\mu$$ (59)

$$= \frac{1}{\sqrt{2\pi}\,\sqrt{\sigma_p^2+\sigma_f^2}} \,\exp \left[-\frac{(d_f-d_p)^2}{2\,(\sigma_p^2+\sigma_f^2)}\right] \,.$$ (60)