Role of priors

Applying the updating reasoning to our box game, the Bayes factor of interest is
$\displaystyle \tilde O_{1,2}(W,I)$ $\displaystyle =$ $\displaystyle \frac{P(W\,\vert\,B_1,I)}{P(W\,\vert\,B_2,I)}
= \frac{1}{1/13}= 13\,.$ (7)

As it was remarked, this number would give the required odds if the hypotheses were initially equally likely. But how strong are the initial relative beliefs on the two hypotheses? `Unfortunately', we cannot perform a probabilistic inversion if we are unable to assign somehow prior probabilities to the hypotheses we are interested in.9
Figure: Graphical representation of the causal connections Box $ \rightarrow E_i$, where $ E_i$ are the effects (White/Black at each extraction). These effects are then causes of other effects ($ E_i\_T$), the reported colors, where `$ T$' stands for `testimony'. The arrows connecting the various `nodes' represent conditional probabilities. The model will be fully exploited in Appendix J.
Indeed, in the formulation of the problem I on purpose passed over the relevant pieces of information to evaluate the prior probabilities (it was said that ``there are two types of boxes'', not ``there are two boxes''!). If we specify that we had $ n_1$ boxes of type $ B_1$ and $ n_2$ of the other kind, then the initial odds are $ n_1/n_2$ and the final ones will be
$\displaystyle O_{1,2}(W,I)$ $\displaystyle =$ $\displaystyle \tilde{O}_{1,2}(W,I)\times O_{1,2}(I)$ (8)
  $\displaystyle =$ $\displaystyle 13\times \frac{n_1}{n_2},$ (9)

from which we get (just requiring that the probability of the two hypotheses have to sum up to one10)
$\displaystyle P(B_1\,\vert\,W,I_0)$ $\displaystyle =$ $\displaystyle \frac{13}{13+n_2/n_1}\,.$ (10)

If the two hypotheses were initially considered equally likely, then the evidence $ W$ makes $ B_1$ 13 times more believable than $ B_2$, i.e. $ P(B_1\,\vert\,W,I_0)=13/14$, or approximately 93%. On the other hand, if $ B_1$ was a priori much less credible than $ B_2$, for example by a factor 13, just to play with round numbers, the same evidence made $ B_1$ and $ B_2$ equally likely. Instead, if we were initially in strong favor of $ B_1$, considering it for instance 13 times more plausible than $ B_2$, that evidence turned this factor into 169, making us 99.4% confident - highly confident, some would even say `practically sure'! - that the box is of type $ B_1$.

Giulio D'Agostini 2010-09-30