Appendix - A case of possibly `too good' agreement

Let us also see a case in which the mutual agreement among individual results `seems' too good. In order to use again the kaon mass data, we take the four results published before year 1988, shown with solid blue Gaussians in Fig. 16, to which we over-impose (usual dashed red Gaussian) the outcome of the weighted average yielding $493.664 \pm 0.015\,$MeV. But in this case our suspicion is that the uncertainty could be overestimated. Indeed, if we calculate the $\chi^2$ we get 0.818, with a $\chi^2/\nu$ of 0.27 (p-value 0.85). Applying strictly the $\sqrt {\chi ^2/\nu }$ scaling prescription - frequentist gurus probably might not agree, but let us go on with the exercise - we get a scaling factor of $\times 0.52$, and thus an `error' of $7.7\,$keV (dotted gray Gaussian). The posterior pdf of the sceptical analysis (solid thick black line) is this time practically Gaussian and gives $493.664\pm 0.012\,$MeV: the curve is narrower than the simple weighted average, in agreement with our suspicions, but not as narrow as when the $\sqrt {\chi ^2/\nu }$ scaling was (improperly?) used. Conclusions on this last comparisons are left to the reader.

Figure: Standard and sceptical analysis of the results published before year 1988.