Bayes theorem and Bayes factor

The `probabilistic inversion' $P(W\,\vert\,B_1,I) \rightarrow P(B_1\,\vert\,W,I)$ can only³ be performed using the so-called Bayes' theorem, a simple consequence of the fact that, given the effect $E$ and some hypotheses $H_i$ concerning its possible cause, the joint probability of $E$ and $H_i$, conditioned by the background information⁴ $I$, can be written as

$$P(E\cap H_i\,\vert\,I) = P(E\,\vert\,H_i,I)\cdot P(H_i\,\vert\,I) = P(H_i\,\vert\,E,I)\cdot P(E\,\vert\,I)\,,$$ (2)

where `$\cap$' stands for a logical `AND'. From the second equality of the last equation we get

$$P(H_i\,\vert\,E,I) = \frac{P(E\,\vert\,H_i,I)}{P(E\,\vert\,I)}\cdot P(H_i\,\vert\,I)\,,$$ (3)

that is one of the ways to express Bayes' theorem.⁵

Since a similar expression holds for any other hypothesis $H_j$, dividing member by member the two expressions we can restate the theorem in terms of the relative beliefs, that is

$$\underbrace{\frac{P(H_i\,\vert\,E,I)}{P(H_j\,\vert\,E,I)}}_{\mbox{updated odds}} = \underbrace{\frac{P(E\,\vert\,H_i,I)}{P(E\,\vert\,H_j,I)} }_{\beg... ...\underbrace{\frac{P(H_i\,\vert\,I)}{P(H_j\,\vert\,I)}}_{\mbox{initial odds}}\,:$$ (4)

the initial ratio of beliefs (`odds') is updated by the so-called Bayes factor, that depends on how likely each hypothesis can produce that effect.⁶Introducing $O_{i,j}$ and BF$_{i,j}$, with obvious meanings, we can rewrite Eq. (4) as

$$O_{i,j}(E,I) = _{i,j}(E,I) \times O_{i,j}(I)\,.$$ (5)

Note that, if the initial odds are unitary, than the final odds are equal to the updating factor. Then, Bayes factors can be interpreted as odds due only to an individual piece of evidence, if the two hypotheses were considered initially equally likely.⁷ This allows us to rewrite BF$_{i,j}(E,I)$ as $\tilde{O}_{i,j}(E,I)$, where the tilde is to remind that they are not properly odds, but rather `pseudo-odds'. We get then an expression in which all terms have virtually uniform meaning:

$$O_{i,j}(E,I) = \tilde{O}_{i,j}(E,I) \times O_{i,j}(I)\,.$$ (6)

If we have only two hypotheses, we get simply $O_{1,2}(E,I) = \tilde{O}_{1,2}(E,I) \times O_{1,2}(I)$. If the updating factor is unitary, then the piece of evidence does not modify our opinion on the two hypotheses (no matter how small can numerator and denominator be, as long as their ratio remains finite and unitary! - see Appendix G for an example worked out in details); when $\tilde{O}_{1,2}(E,I)$ vanishes, then hypothesis $B_1$ becomes impossible (``it is falsified''); if instead it is infinite (i.e. the denominator vanishes), then it is the other hypothesis to be impossible. (The undefined case $0/0$ means that we have to look for other hypotheses to explain the effect.⁸)