Further scepticism

If the purpose of this paper would have been just to search around for a case of `apparent' discordant results, as a real life example to which apply to the model of Ref.[15] implemented in JAGS, then the game would be at the end. But since I am presently interested in the charged kaon mass, I tried to understand the results a bit more. I expected in fact to find in the publications extensive discussions on the details of the analysis, with explanations of what the results really meant and detailed accounts for the sources of uncertainties, as it has presently become a good practice by most experimental teams. But this was not the case. Already trying to understand the (apparently, as we shall see) most precise value, I was quite surprised when I realized that Ref. [9] gives no detailed information on how they got their numbers and on what their `error' really means. Furthermore the PDG uses an `error' of 0.007 MeV, instead of the 0.0059 MeV reported by [9], on the basis of a PhD thesis [10] which it is impossible to find (not even in Russian!). Fortunately this is more a methodological paper then a real attempt to get a deep understanding of the charged kaon mass, for which a throughout analysis of all relevant published matter on the subject would be required.²¹

Nevertheless, there is a point I would like to touch, related to the second most precise result of the list [8] whose conservative uncertainty is uncritically accepted by the PDG. The paper provides in fact four mass values, reported for the reader's convenience in Fig. 12.

Figure: Details of Ref. [8].

The weighted average is $493.6355 \pm 0.0067$, rescaled (and rounded) to $493.636 \pm 0.011$ based on a `high $\chi^2$', which is in reality not so bad, being 7.0 with 3 degrees of freedom,²² and thus yielding a p-value of 0.072, above even the (in-)famous threshold of 0.05 [32]. For this reason I could not resist to make a couple of exercises: first to see what a sceptical analysis would suggest if we stick to the simple weighted average, without the $\sqrt{2.31}\ (=1.52)$ scaling; second to see what we get if we make an overall sceptical analysis in which individual results are used.