Evidences mediated by a testimony

In most cases (and practically always in courts) pieces of evidence are not acquired directly by the person who has to form his mind about the plausibility of a hypothesis. They are usually accounted by an intermediate person, or by a chain of individuals. Let us call $E_T$ the report of the evidence $E$ provided in a testimony. The inference becomes now $P(H_i\,\vert\,E_T,I)$, generally different from $P(H_i\,\vert\,E,I)$.

In order to apply Bayes' theorem in one of its form we need first to evaluate $P(E_T\,\vert\,H_i,I)$. Probability theory teaches us how to get it [see Eq. (33) in Appendix A]:

$$P(E_T\,\vert\,H_i,I) = P(E_T\,\vert\,E,I)\cdot P(E\,\vert\,H_i,I) + P(E_T\,\vert\,\overline E,I)\cdot P(\overline E\,\vert\,H_i,I)$$ (38)

($E_T$ could be due to a true evidence or to a fake one). Three new ingredients enter the game:

• $P(E_T\,\vert\,E,I)$, that is the probability of the evidence to be correctly reported as such.
• But the testimony could also be incorrect the other way around (it could be incorrectly reported, simply by mistake, but also it could be a `fabricated evidence'), and therefore also $P(E_T\,\vert\,\overline E,I)$ is needed. Note that the probabilities to lie could be in general asymmetric, i.e. $P(\overline E_T\,\vert\,E,I)\ne P(E_T\,\vert\,\overline E,I)$, as we have seen in the AIDS problem of Appendix F, in which the response of the analysis models false witness well.
• Finally, since $P(\overline E\,\vert\,H_i,I)$ enters now directly, the `naïve' Bayes factor, only depending on $P(E\,\vert\,H_i,I)$, is not longer enough.

Taking our usual two hypotheses, $H_1=H=$`guilty' and $H_2=\overline{H}=$`innocent', we get the following Bayes factor based on the testified evidence $E_T$ (hereafter, in order to simplify the notation, we use the subscript `$H$' in odds and Bayes factors, instead of `$i,j$', to indicate that they are in favor of $H$ and against $\overline H$, as we already did in the AIDS example of Appendix F):

$$\tilde O_H(E_T,I) = \frac{ P(E_T\,\vert\,E,I)\cdot P(E\,\vert\,H,I) + P(E_T\,\vert\,\... ...) + P(E_T\,\vert\,\overline E,I)\cdot P(\overline E\,\vert\,\overline H,I) }\,.$$ (39)

As expected, this formula is a bit more complicate that the Bayes factor calculated taking $E$ for granted, which is recovered if the lie probabilities vanish

$$\tilde O_{H}(E_T,I) \xrightarrow[ \mbox{{\footnotesize $ P(E_T\,\vert\,\overline E,I)\rightarrow 0$}} ]{} {\ \ \ \tilde O_{H}(E,I)}\,,$$ (40)

i.e. only when we are absolutely sure the witness does not err or lie reporting $E$ (but Peirce reminds us that ``absolute certainty, or an infinite chance, can never be attained by mortals'' [6]).

In order to single out the effects of the new ingredients, Eq. (39) can be rewritten as⁶²

$$\tilde O_{H}(E_T,I) = \tilde O_{H}(E,I)\times \frac{1 + \lambda(I) \cdot \left[ \frac{1... ...bda(I) \cdot \left[ \frac{ \tilde O_{H}(E,I)}{P(E\,\vert\,H,I)} -1 \right]} \,,$$ (41)

where

$$\lambda(I) = \frac{P(E_T\,\vert\,\overline E,I)}{P(E_T\,\vert\,E,I)}\,,$$ (42)

under the condition⁶³ $P(E\,\vert\,H,I)> 0$ and $P(E\,\vert\,\overline H,I)> 0$, i.e. $\tilde O_{H}(E,I)$ positive and finite. The parameter $\lambda(I)$, ratio of the probability of fake evidence and the probability that the evidence is correctly accounted, can be interpreted as a kind of lie factor. Given the human roughly logarithmic sensibility to probability ratios, it might be useful to define, in analogy to the JL,

$$\lambda(I) = \log_{10}[\lambda(I)]\,.$$ (43)

Let us make some instructive limits of Eq. (41).

$$\tilde O_{H}(E_T,I) \xrightarrow[ \mbox{{\footnotesize $\lambda(I) \rightarrow 0$}} ]{}{} \tilde O_{H}(E,I)$$ (44)

$$\tilde O_{H}(E_T,I) \xrightarrow[ \mbox{{\footnotesize $\lambda(I) \rightarrow 1$}} ]{}{} 1$$ (45)

$$\tilde O_{H}(E_T,I) \xrightarrow[ \mbox{{\footnotesize $P(E\,\vert\,H,I)\rightarrow 0$}} ]{}{} 1$$ (46)

$$\tilde O_{H}(E_T,I) \xrightarrow[ \mbox{{\footnotesize $\tilde O_{H}(E,I)\rightarrow \infty $}} ]{}{} \frac{P(E\,\vert\,H,I)}{\lambda(I)} + 1 - P(E\,\vert\,H,I)$$ (47)

As we have seen, the ideal case is recovered if the lie factor vanishes. Instead, if it is equal to 1, i.e. J$\lambda(I)=0$, the reported evidence becomes useless. The same happens if $P(E\,\vert\,H,I)$ vanishes [this implies that $P(E\,\vert\,\overline H,I)$ vanishes too, being $P(\overline H,I)=P(E\,\vert\,\overline H,I)/\tilde O_{H}(E,I)$].

However, the most remarkable limit is the last one. It states that, even if $\tilde O_{H}(E,I)$ is very high, the effective Bayes factor cannot exceed the inverse of the lie factor:

$$\tilde O_{H}(E_T,I) \le \frac{P(E\,\vert\,H,I)}{\lambda(I)} \le \frac{1}{\lambda} \mbox{[if $\tilde O_{H}(E,I)\rightarrow\infty$]} \,,$$ (48)

or, using logarithmic quantities

$$\Delta (E_T,I) \le - \lambda + \log_{10}{P(E\,\vert\,H,I)} \le - \lambda \mbox{[if $\Delta$JL$(E,I)\rightarrow\infty$]} \,.$$ (49)

At this point some numerical examples are in order (and those who claim they can form their mind on pure intuition get all my admiration...if they really can). Let us imagine that $E$ would ideally provide a weight of evidence of 6 [i.e. $\Delta$JL$_H(E,I)=6$]. We can study, with the help of table 2,

Table: Dependence of the judgement leaning due to a reported evidence [ $\Delta$JL$_H(E_T,I)$] for $\Delta$JL$_H(E,I)=6$, 3 and 1 as a function the other ingredients of the inference (see text). Note the upper limit of $\Delta$JL$_H(E_T,I)$ to $-$J$\lambda$, if this value is $\le \Delta$JL$_H(E,I)$.

$\Delta$JL$_H(E,I)=6$
J$\lambda(I)$		JL$_H(E_T,I)$
	JL$_{E}(H,I)$:	10	$3$	$2$	$1$	0	$-1$	$-3$	$-10$
$\rightarrow -\infty$		6.00	6.00	6.00	6.00	6.00	6.00	6.00	6.00
$-8$		6.00	6.00	6.00	6.00	5.99	5.95	4.96	$4\times 10^{-3}$
$-7$		5.96	5.96	5.96	5.95	5.92	5.68	4.00	$4\times 10^{-4}$
$-6$		5.70	5.70	5.70	5.68	5.52	4.92	3.00	$4\times 10^{-5}$
$-5$		4.96	4.96	4.95	4.92	4.68	3.95	2.00	$4\times 10^{-6}$
$-4$		4.00	4.00	3.99	3.95	3.70	2.96	1.04	$4\times 10^{-7}$
$-3$		3.00	3.00	3.00	2.96	2.70	1.96	0.30	$4\times 10^{-8}$
$-2$		2.00	2.00	2.00	1.95	1.70	1.00	0.04	$4\times 10^{-9}$
$-1$		1.00	1.00	1.00	0.96	0.74	0.26	0.004	$4\times 10^{-10}$
0		0	0	0	0	0	0	0	0
$\Delta$JL$_H(E,I)=3$
J$\lambda(I)$		JL$_H(E_T,I)$
	JL$_{E}(H,I)$:	10	$3$	$2$	$1$	0	$-1$	$-3$	$-10$
$\rightarrow -\infty$		3.00	3.00	3.00	3.00	3.00	3.00	3.00	3.00
$-5$		3.00	3.00	3.00	3.00	2.99	2.95	1.96	$4\times 10^{-6}$
$-4$		2.96	2.96	2.96	2.95	2.92	2.68	1.04	$4\times 10^{-7}$
$-3$		2.70	2.70	2.70	2.68	2.52	1.93	0.30	$4\times 10^{-8}$
$-2$		1.96	1.96	1.96	1.92	1.68	1.00	0.04	$4\times 10^{-9}$
$-1$		1.00	1.00	0.99	0.96	0.74	0.26	0.004	$4\times 10^{-10}$
0		0	0	0	0	0	0	0	0
$\Delta$JL$_H(E,I)=1$
J$\lambda(I)$		JL$_H(E_T,I)$
	JL$_{E}(H,I)$:	10	$3$	$2$	$1$	0	$-1$	$-3$	$-10$
$\rightarrow -\infty$		1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
$-3$		1.00	1.00	1.00	1.00	0.99	0.96	0.26	$4\times 10^{-8}$
$-2$		0.96	0.96	0.96	0.96	0.93	0.72	0.04	$4\times 10^{-9}$
$-1$		0.72	0.72	0.72	0.70	0.58	0.23	0.003	$4\times 10^{-10}$
$-0.5$		0.41	0.41	0.41	0.39	0.27	0.07	$8\times 10^{-4}$	$4\times 10^{-10}$
0		0	0	0	0	0	0	0	0

how the weight of the reported evidence $\Delta$JL$_H(E_T,I)$ depends on the other beliefs [in this table logarithmic quantities have been used throughout, therefore JL$_E(H,I)$ is the base ten logarithm of the odds in favor of $E$ given the hypothesis $H$; the table provides, for comparisons, also $\Delta$JL$_H(E_T,I)$ from $\Delta$JL$_H(E,I)$ equal to 3 and 1].

The table exhibits the limit behaviors we have seen analytically. In particular, if we fully trust the report, i.e. J$\lambda(I)=-\infty$, then $\Delta$JL$_H(E_T,I)$ is exactly equal to $\Delta$JL$_{H}(E,I)$, as we already know. But as soon as the absolute value of the lie factor is close to JL$_H(E,I)$, there is a sizeable effect. The upper bound can be the be rewritten as

$$\tilde O_{H}(E_T,I) \le \, [\tilde O_{H}(E,I),\, \frac{1}{\lambda}]\,,$$ (50) or     

$$\Delta _H(E_T,I) \le \,[\Delta _H(E,I),\,- \lambda(I)] \,,$$ (51)

a relation valid in the region of interest when thinking about an evidence in favor of $H$, i.e. $\Delta$JL$_H(E,I) > 0$ and J$\lambda(I) < 0$.

This upper bound is very interesting. Since minimum conceivable values of J$\lambda(I)$ for human beings can be of the order of $-6$ (to perhaps $\approx -8$ or $\approx -9$, but in many practical applications $-2$ or $-3$ can already be very generous!), in practice the effective weights of evidence cannot exceed values of about $+6$ (I have no strong opinion on the exact value of this limit, my main point is that you consider there might be such a practical human limit.)

This observation has an important consequence in the combination of evidences, as anticipated at the end of section 3.5. Should we give more consideration to a single strong piece of evidence, virtually weighing $\Delta$JL$(E)=10$, or 10 independent weaker evidences, each having a $\Delta$JL of 1? As it was said, in the ideal case they yield the same global leaning factor. But as soon as human fallacy (or conspiracy) is taken into account, and we remember that our belief is based on $E_T$ and not on $E$, then we realize that $\Delta$JL$(E_T)=10$ is well above the range of JL that we can reasonably conceive. Instead the weaker pieces of evidence are little affected by this doubt and when they sum up together, they really can provide a $\Delta$JL of about 10.