Introduction

Our perspectives about living with Covid-19 are changed dramatically in the last months of 2020 with the results from the vaccine trials by Pfizer and Moderna and, a bit later, by AstraZeneca. The first one claimed a 90% efficacy [2] (then updated to 95% in a further press release [3] and in a much more detailed paper [4]) and the second one 94.5% [5] (later updated to 94.1% [6,7]), while AstraZeneca press released and then published two values (90.0% and 62.1%) depending on two (unplanned [8,9]) different experimental settings.¹

In an unpublished first paper² based on the first press releases by Pfizer and Moderna, we remarked that, since the announcements did not mention any uncertainty, we understood that the initial Pfizer's number was the result of a rounding, with uncertainty of the order of the percent. But then, we continued, we were highly surprised by the Moderna's announcement, providing the tenths of the percent, as if it were much more precise. We had indeed the impression that the `point five' was taken very seriously, not only by media speakers, who put the emphasis on the third digit, but also by experts from which we would have expected a phrasing implying some uncertainty in the result (see e.g. Ref. [11]). The same remarks apply to the later AstraZeneca announcement. In fact, a fast exercise shows that, in order to have an uncertainty of the order of a few tenths of percent, the number of vaccine-treated individuals that got the Covid-19 had to be at least of the order of several hundreds. But this was not the case. In fact, the actual numbers were indeed much smaller, as we learned from the Moderna first press release [5]: “This first interim analysis was based on 95 cases, of which 90 cases of COVID-19 were observed in the placebo group versus 5 cases observed in the mRNA-1273 group, resulting in a point estimate of vaccine efficacy of 94.5% ($p <0.0001$)”.

Now, it is a matter of fact that if a physicist reads for an experimental result a number like `5', she tends to associate to it, as a rule of thumb, an uncertainty of the order of its square root, that is $\approx 2.2$. Applied to the Moderna claims, this implies an inefficacy of about $(5.5\pm 2.3)\%$, or an efficacy of about $(94.5\pm 2.3)\%$. Another reason that made us worry about the result was, besides the absence of an associated uncertainty [12], the “$p <0.0001$” accompanying it, being us extremely critical against p-values and other frequentist prescriptions (see Ref. [13] and references therein).

Table: Bare data concerning the number of infected in the vaccine group ($n_{V_I}$) and in the placebo group ($n_{P_I}$). `Moderna-1' is just an interim result, based on the same sample of `Moderna-2', that we had used in Ref. [10]. In the case of Moderna and Pfizer comprehensive results, also the numbers for the occurrence of `severe forms of infection' were reported ( $n_{V_{I_s}}$ and $n_{P_{I_s}}$, respectively). In the case of AstraZeneca LD-SD stands for `low dose followed by standard dose', SD-SD for `two consecutive standard doses'.

	release date	sample size vaccine - placebo	$n_{V_I}$	$n_{P_I}$	$n_{V_{I_s}}$	$n_{P_{I_s}}$
Moderna-1 [5]	Nov 16	14134 - 14073	5	90	-	-
Moderna-2 [6,7]	Nov 30	(same)	11	185	0	30
Pfizer [3,4]	Nov 18	18198 - 18325	8	162	1	9
AstraZeneca LD-SD [8,9]	Nov 23	1367 - 1374	3	30	-	-
AstraZeneca SD-SD [8,9]	Nov 23	4440 - 4455	27	71	-	-

Table: Published vaccine efficacy (value and 95% `uncertainty interval').

	efficacy value	95% `uncertainty interval'
Moderna-1 [5]	$0.945$	-----
Moderna-2 [7]	$0.941$	$[0.893, 0.968]$ (confidence interval)
Pfizer [4]	$0.950$	$[0.903, 0.976]$ (credible interval)
AstraZeneca LDSD [8]	$0.900$	$[0.674, 0.970]$ (confidence interval)
AstraZeneca SDSD [8]	$0.621$	$[0.410, 0.757]$ (confidence interval)

In our first paper [10] we tried then to understand whether it was possible to get an idea of the possible values of efficacy consistent with the data, each one associated with its degree of belief on the basis of the few data available in those days. In other words, our purpose was and is to arrive to a probability density function (pdf), although not obtained in closed form, of the quantity of interest.

In the present paper we not only extend our analysis to the published data [4,7,9] (see Tab. ), but can also compare our results with the published ones (see Tab. ), which also include an indication of the uncertainty to be associated with them. What makes us confident about the validity of our simple model is that the press released and finally published results concerning `efficacy' (see Tab. ) are in excellent agreement with the mode of the distribution we get analyzing our model through a Markov Chain Monte Carlo (MCMC). This is not a surprise to us, indeed. In fact we are aware of statistical methods which tend to produce as `estimate' the most probable value of the quantity of interest, that would be inferred starting from a flat prior [14]. The fact that different kind of `uncertainty intervals' are provided will be discussed at the due point. We only anticipate here that they have in this case equivalent meaning.

The paper is organized as follows. In Sec.  we describe and show how to implement in JAGS [15] the causal model connecting in a probabilistic way the quantities of interest, among which the primary role is played by the `efficacy' $\epsilon$. We also give, in footnote , some indications on how to proceed in order to get exact results for $f(\epsilon)$, although they can only be obtained numerically. The MCMC results are shown and discussed in Sec. . Then the question asked in the title is tackled, with a didactic touch and including some historical remarks, in Sec. . The observation that the resulting pdf's of $\epsilon$ can by approximated rather well by Beta distributions (Sec. ) leads us to discuss in further detail the role of the priors, initially chosen simply uniform. Then Sec.  is devoted to the related question of predicting the number of vaccinated people that shall result infected, taking into account several sources of uncertainty. Finally, in Sec.  we extend our analysis to the level of protection given by the vaccines against the disease severity, in which the outcome of a simple application of probability theory is at odds with simplistic, extraordinary claims.³In Sec.  we sum up the analysis strategy and the outcome of the paper. Then some conclusions and final remarks follow.