The statistically motivated claim

A customary way to quantify the difference between an observed spectrum and the expected one is the famous $\chi^2$ statistic.⁶ The CDF paper reports a ``$\chi^2$ per degree of freedom'' ($\chi^2/\nu$) of 77.1/84 for the entire spectrum and 26.1/20 for the region 120-160 GeV. In both cases statistical practice based on this test states that ``there is nothing to be surprised''.

I know by experience that, when a test does not say what practitioners would like, other tests are tried - like when one goes around looking for someone that finally says one is right.⁷ Indeed, in the statistics practice there is much freedom and arbitrariness about which test to use and how to use it. This is because hypothesis tests of the so called classical statistics do not follow strictly from probability theory, but are just a collections of ad hoc prescriptions. For this reason I do not want to enter on what CDF finally quotes as p-value (with the only comment that it does not even seem a usual p-value). Let us then just stick to the paper, reporting here the claim, followed by a reminder about what a statistician would understand by that name:

• ``we obtain a p-value of $7.6\times 10^{-4}$, corresponding to a significance of 3.2 standard deviations'';[9]
• ``the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.''[11] [The null hypothesis ($H_0$) is in this case ``only standard physics, without contributions from new phenomena''.]