Probability of observations vs probability of the values of physical quantities

The third section of `book 2' of the Gauss' tome [7] is dedicated to “the determination of an orbit satisfying as nearly as possible any number of observations whatever”.⁴After `articles' ⁵ 172-174, which introduce the specific problem of evaluating the elements of an orbit from the measurements of geocentric quantities related to those elements, with article 175 Gauss ascends ⁶to methodological issues of general interest for the Sciences:

“let us leave our special problem, and enter upon a very general discussion and one of the most fruitful in every application of the calculus to the natural philosophy.”

The general problem is how to determine the $\mu$ unknown quantities

, etc. (e.g. the elements of the orbit of a planet or a comet) and evaluate the functions

of these variables from $\nu$ measurements $V_{m_i}$ (e.g. the geocentric quantities of that celestial body measured at different times):⁷

$\displaystyle V_i(p,q,r,s,\dots)$

$\displaystyle \xrightarrow[$ measured as $\displaystyle ]{}$

$\displaystyle V_{m_i}\,,$

(4)

or, indicating the set of unknown quantities by $\theta$ , that is $\theta$ $= \{p,q,r,s,\dots\}$ , we can rewrite Eq. (4) as

$\displaystyle V_i($ $\displaystyle \mbox{\boldmath$\theta$}$ $\displaystyle )$

$\displaystyle \xrightarrow[$ measured as $\displaystyle ]{}$

$\displaystyle V_{m_i} \,.$

(5)

The most interesting case, Gauss explains, is when $\nu > \mu$ . Being over-determined, this case has a solution only if $V_{m_i}$ are affected by experimental errors, described by the probability density function⁸(pdf) $\varphi$ , that in article 177 will came out to be the well known Gaussian function.⁹Therefore,¹⁰

“Supposing, therefore, any determinate system of the values of the quantities , , , , etc., the probability that the observation would give for the value will be expressed by $\varphi(M\!-\!V)$ , substituting in for , , , , etc., their values; in the same manner $\varphi(M'\!-\!V')$ , $\varphi(M''\!-\!V'')$ , etc, will express the probability that observation would give the values , , etc. of the functions , , etc. Wherefore, since we are authorized to regard all observations as event independent of each other, the product

$\displaystyle \hspace{2.0cm}\varphi(M\!-\!V)\,\varphi(M'\!-\!V')\, \varphi(M''\!-\!V'')\,$ etc, $\displaystyle \, = \,\Omega$ (G1)

will express the expectation or probability that all those values will result together from observation.”

What Gauss calls $\Omega$ is thus the joint pdf of the differences $V_{m_i}-V_i$ given a precise set of values for the physical quantities of interest, which we would rewrite as

$\displaystyle f($ $\displaystyle \mbox{\boldmath$V_m$}$ $\displaystyle -\!$ $\displaystyle \mbox{\boldmath$V$}$ $\displaystyle \,\vert\,$ $\displaystyle \mbox{\boldmath$\theta$}$ $\displaystyle )$

$\displaystyle =$

$\displaystyle \prod_i\varphi(V\!_{m_i}\!-\!V_i\,\vert\,$ $\displaystyle \mbox{\boldmath$\theta$}$ $\displaystyle )$

(6)

where and stand for the set of observations and of functions.¹¹

Article 175 ends so with the expression of the joint probability of the observations given any set of values of the quantities of interest, that is a problem in direct probabilities:

$\theta$ $\displaystyle \xrightarrow[$ deterministic link $\displaystyle ]{}$ $\displaystyle \xrightarrow[$ probabilistic link $\displaystyle ]{}$ $\displaystyle$

Article 176 begins with what we could call nowadays a `Bayesian manifesto':

“Now in the same manner as, when any determinate values whatever of the unknown quantities being taken, a determinate probability corresponds, previous to observations, to any system of values of the functions , , , etc; so, inversely, after determinate values of the functions have resulted from observation, a determinate probability will belong to every system of values of the unknown quantities, from which the values of the functions could possibly have resulted.”

That is, in our notation, as when we assume “determinate values” of the physical quantities we are interested in the joint pdf of the values that will be observed,

$\displaystyle f($ $\displaystyle \mbox{\boldmath$V_m$}$ $\displaystyle -\!$ $\displaystyle \mbox{\boldmath$V$}$ $\displaystyle \,\vert\,$ $\displaystyle \mbox{\boldmath$\theta$}$ $\displaystyle )\,,$

(7)

similarly, once the observations have been made, we are interested in the joint pdf of the values of the physical quantities,

$\displaystyle f($ $\displaystyle \mbox{\boldmath$\theta$}$ $\displaystyle \,\vert\,$ $\displaystyle \mbox{\boldmath$V_m$}$ $\displaystyle -\!$ $\displaystyle \mbox{\boldmath$V$}$ $\displaystyle )\,.$

(8)

The question is now how to go from Eq. (7) to Eq. (8), reasoning “inversely”.