Role of priors

Applying the updating reasoning to our box game, the Bayes factor of interest is

$$\tilde O_{1,2}(W,I) = \frac{P(W\,\vert\,B_1,I)}{P(W\,\vert\,B_2,I)} = \frac{1}{1/13}= 13\,.$$ (7)

As it was remarked, this number would give the required odds if the hypotheses were initially equally likely. But how strong are the initial relative beliefs on the two hypotheses? `Unfortunately', we cannot perform a probabilistic inversion if we are unable to assign somehow prior probabilities to the hypotheses we are interested in.⁹

Figure: Graphical representation of the causal connections Box $\rightarrow E_i$, where $E_i$ are the effects (White/Black at each extraction). These effects are then causes of other effects ($E_i\_T$), the reported colors, where `$T$' stands for `testimony'. The arrows connecting the various `nodes' represent conditional probabilities. The model will be fully exploited in Appendix J.

Indeed, in the formulation of the problem I on purpose passed over the relevant pieces of information to evaluate the prior probabilities (it was said that ``there are two types of boxes'', not ``there are two boxes''!). If we specify that we had $n_1$ boxes of type $B_1$ and $n_2$ of the other kind, then the initial odds are $n_1/n_2$ and the final ones will be

$$O_{1,2}(W,I) = \tilde{O}_{1,2}(W,I)\times O_{1,2}(I)$$ (8)

$$= 13\times \frac{n_1}{n_2},$$ (9)

from which we get (just requiring that the probability of the two hypotheses have to sum up to one¹⁰)

$$P(B_1\,\vert\,W,I_0) = \frac{13}{13+n_2/n_1}\,.$$ (10)

If the two hypotheses were initially considered equally likely, then the evidence $W$ makes $B_1$ 13 times more believable than $B_2$, i.e. $P(B_1\,\vert\,W,I_0)=13/14$, or approximately 93%. On the other hand, if $B_1$ was a priori much less credible than $B_2$, for example by a factor 13, just to play with round numbers, the same evidence made $B_1$ and $B_2$ equally likely. Instead, if we were initially in strong favor of $B_1$, considering it for instance 13 times more plausible than $B_2$, that evidence turned this factor into 169, making us 99.4% confident - highly confident, some would even say `practically sure'! - that the box is of type $B_1$.