... D'Agostini¹

Università “La Sapienza” and INFN, Roma, Italia, giulio.dagostini@roma1.infn.it

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... Esposito²

Retired, alfespo@yahoo.it

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... settings.¹

In one of the experimental setting, indicated here as `low dose'-`standard dose', the first vaccine dose was half of the planned one (`standard dose'-`standard dose' setting).

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... paper²

The paper is available on the web site of one of the authors, together with the slides of a related webinar and the code to reproduce the results [12].

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... claims.³

 `All 30 cases occurred in the placebo group and none in the mRNA-1273 vaccinated group.' `...and vaccine efficacy against severe COVID-19 was 100%' (Moderna press release [8], based on no severe cases out of the 11 infectees in the vaccine group). This result has been reported as 100% efficacy with (uncritical!) great emphasis also in the media [18] – a reminder of the C. Sagan's quote that “extraordinary claims require extraordinary evidence” is here in order.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... techniques⁴

Indeed one could try to get an exact solution for the pdf of $\epsilon$. The steps needed are: write down the joint pdf of all the variables in the network; condition on the certain variables; marginalize over all the uncertain variables besides $\epsilon$. Referring to Ref. [19] for details, here is the structure of the unnormalized pdf obtained starting from uniform priors over $\epsilon$ and $p_A$:

$$f(\epsilon\,\vert\,n_V,n_P,n_{V_I},n_{P_I}) \propto \sum_{n_{V_A}=n_{V_I}}^{n_V}\int_0^1\! p_A \left[\frac{n_{V_A}!}{(n_{V_A}-n_{V_I})!}\cdot(1-\epsilon)^{n_{V_I}}\cdot \epsilon^{n_{V_A}-n_{V_I}}\right] \cdot$$

$$\hspace{2.05cm} \left[\frac{1}{n_{V_A}!\,(n_V-n_{V_A})!}\cdot p_A^{n_{V_A}}\cdot (1-p_A)^{n_V-n_{V_A}}\right] \cdot$$

$$\hspace{2.05cm}\left[p_A^{n_{P_A}}\cdot (1-p_A)^{n_P-n_{P_A}}\right] \,,$$

where the three terms within square brackets are the three binomial distributions entering the model, stripped of all irrelevant constant factors. Simplifying and reorganizing the various terms we get

$$f(\epsilon\,\vert\,n_V,n_P,n_{V_I},n_{P_I}) \propto (1-\epsilon)^{n_{V_I}}\cdot \sum_{n_{V_A}=n_{V_I}}^{n_V}\frac{\epsilon^{n_{V_A}-n_{V_I}}} {({n_{V_A} -n_{V_I})}!\,(n_V-n_{V_A})!}\cdot$$

$$\int_0^1 p_A^{n_{V_A}+{n_{P_A}}}\cdot (1-p_A)^{n_V-n_{V_A}+{n_P-n_{P_A}}}\, p_A\,.$$

We recognize that the integral $\int_0^1x^{r-1}\cdot (1-x)^{s-1}\,$d$x$, in terms of a generic variable $x$, defines the special function beta  B$(r,s)$, thus obtaining

$$f(\epsilon\,\vert\,\ldots) \!\! \propto (1-\epsilon)^{n_{V_I}}\cdot\!\!\! \sum_{n_{V_A}=n_{V_I}}^{n_V}\fr... ...n^{n_{V_A}\!-n_{V_I}}} {({n_{V_A}\! -n_{V_I})}!\,(n_V\!-n_{V_A})!} \cdot % \\ (n_{V_A}\!\!+\!{n_{P_A}}\!\!+\!1,\, n_V\!\!-\!n_{V_A}\!+n_P\!-\!n_{P_A}\!\!+\!1).$$

Then the integral over $\epsilon$ follows, in order to get the normalization factor. Finally, all moments of interest can be evaluated. All this can be done numerically. However, we proceed to MCMC, being its use much simpler and also for the flexibility it offers (for example in the case we need to extend the model, as we shall do in Secs.  and ).

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... transparent:⁵

Those who have no experience with JAGS can find in Ref. [19] several ready-to-run R scripts.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

....⁶

We cannot go here into the details of this choice that we consider quite reasonable, given the information provided by the data, and refer for the details to Ref. [19] and references therein. The fact that, as we shall see in next section, the modes of the distributions of $\epsilon$ that result from our analysis practically coincide with the efficacy values reported by the three companies means that they have also used `flat priors', or frequentist methods which implicitly entail a flat prior [16]. We shall see in Sec.  how `informative priors' (e.g. by experts) can be taken into account in a second step, without the need of repeating the analysis for each choice of priors.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... interval ⁷

The meaning of such an interval is that, conditioned on the data used and on the model assumptions, we consider $P(\epsilon < \epsilon_{low}) = P(\epsilon > \epsilon_{high}) = 2.5$%, where $\epsilon_{low}$ and $\epsilon_{high}$ are the boundaries of the interval.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... companies.⁸

It is important to understand that, strictly speaking, a 95% frequentistic confidence interval does not provide the interval in which the authors are 95% confident that the `true value' of interest lies (see Refs. [15,16] and references therein), although it is `often the case' for `routine measurements' [16].

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

...).⁹

In other words, the gross result essentially depends on the ratios $n_{V_I}$:$n_{P_I}$ and $n_{V}$:$n_{P}$.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

...hypotheses¹⁰

In general we are used to indicating by $I$ the background state of information [19], but we reserve here the symbol $I$ for `infected'.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... public.¹¹

In this paper we only focus on efficacy, without even trying to enter on the related topics of effectiveness, that refers to how well the vaccine performs in the real world (see e.g. Ref. [22]), that is influenced by several other factors.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... interest.¹²

Let us repeat once more that the result of frequentistic point estimates can be easily shown to be equivalent, under reasonable assumptions, to the mode of the distribution obtained by a probabilistic analysis.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... methodological,¹³

However, we wish to point out that, from the practical point of view, what really matters is the inefficacy i.e. the probability of getting infected (see Sec. for details). Indeed, even though two hypothetical values of $\epsilon$ equal to $0.96$ and $0.98$, respectively, might appear quite close, nevertheless they imply that the relative probabilities of getting infected are in the ratio 2:1.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... future,¹⁴

A reminder of Russell's inductivist turkey is a must at this point!

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... is¹⁵

See e.g. Ref. [19] and references therein.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

...:¹⁶

To make it clear, no `fit' on the MCMC histogram has been performed.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... shown.¹⁷

In fact, assuming that the MCMC based pdf of $\epsilon$ starting from a flat prior (`$F$') can be approximated by a Beta, we can write it, neglecting irrelevant factors, as

$$f_F(\epsilon) \propto \epsilon^{r_F-1}\cdot (1-\epsilon)^{s_F-1}\,.$$

Expressing also the informative prior by a Beta, that is

$$f_0(\epsilon) \propto \epsilon^{r_0-1}\cdot (1-\epsilon)^{s_0-1}\,,$$

and applying the Bayes' rule, we get for the posterior (`$p$')

$$f_p(\epsilon) \propto f_F(\epsilon) \times f_0(\epsilon)$$

$$\propto \epsilon^{r_F-1+r_0-1}\cdot (1-\epsilon)^{s_F-1+s_0-1}$$

$$\propto \epsilon^{(r_F+r_0-1)-1}\cdot (1-\epsilon)^{(s_F+s_0-1)-1}\,,$$

from which Eqs. () - () follow.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... data,¹⁸

There is nothing special with this choice, and what follows is a little more than an exercise, strongly dependent on the assumption on $p'_A$.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... Carlo.¹⁹

For example, here is the R code to be added in the above script, immediately after the assignment `pA <- 0.01', in order to implement Eqs. () - ():

spA <- 0.1 * pA
est.nvI <- nV * pA * (1-mu)
est.sigma.nvI <- sqrt( nV * pA * (1-mu) * (1 - pA * (1-mu)) +
                      (nV*(1-mu))^2 * spA^2 + (nV*pA)^2 * sigma^2 )
cat(sprintf("Approximated nvI: mean+-sigma: %.1f +- %.1f\n",est.nvI,est.sigma.nvI))

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... intervals.²⁰

But this is not a general rule, as discussed in detail in Ref. [16].

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

....²¹

Just to have an idea of the numbers we are dealing with, the values of $\{r_F,\,s_F\!\}$ resulting from our analysis are equal to $\{73,\,5.3\}$, $\{156,\,11\}$, $\{137,\,8.1\}$, $\{17,\,2.8\}$ and $\{17,\,11\}$, respectively for Moderna-1, Moderna-2, Pfizer, AstraZeneca (LDSD) and AstraZeneca (SDSD).

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... her.²²

Remember that Science and its popularization is based on a long chain of rational beliefs [16]. Think for example to the reasons you believe in gravitational waves, provided that you really believe that they could exist and that they have finally being detected on Earth starting from 2015 – our trusted source ensures us that 67 of them have been `observed' so far [1].

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... Carlo.²³

Note also that what really enters in Eqs. () - () is $1\!-\!\epsilon$ whose relative uncertainty is around 30% even in the best cases of Moderna-2 and Pfizer $[$it reaches 54% for AstraZeneca (LDSD), becoming `only' 22% for AstraZeneca (SDSD), characterized however by a large value of $1\!-\!\epsilon$ $]$.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.