Summing up

In this paper, spurred by the press releases first and then by the published results about the performance of the candidate vaccines, we have reanalyzed the published data with the help of a Bayesian network processed with MCMC methods. The aim was that of obtaining, for each data set, the pdf of the model variable $\epsilon$, whose meaning is the following: if we assume for it an exact value, then the probability that a `virus assaulted individual' gets infected would be exactly $1\!-\!\epsilon$.

Our results are in excellent agreement with the published ones, if the latter are properly interpreted. In fact, it turns out that they coincide with the modes of the respective pdf $f(\epsilon)$, although we maintain that if a single number has to be provided, especially to the media, it should be the mean of the distribution. This has, in fact, the meaning of the probability that a vaccinated person will be shielded by the virus, taking into account the unavoidable uncertainty on $\epsilon$, fully described by $f(\epsilon)$. And this is what really matters to define the efficacy of a vaccine. Willing however to reduce the result of our analysis to a single number to be compared with the released ones, we get respectively and with reasonable rounding, 93%, 94%, 94%, 86% and 60% for Moderna-1, Moderna-2, Pfizer, AstraZeneca (LDSD) and AstraZeneca (SDSD), versus 94.5%, 94.1%, 95.0%, 90.0% and 62.1% of Tab. . Therefore, as far as these numbers are concerned, there is then a substantial agreement of the outcome of our analysis with the published results, simply because when a probability distribution is unimodal and rather symmetrical then mode and mean tend to coincide. Therefore, with respect to the main results, our contribution to this point is mainly methodological. The probability theory based result is, instead, at odds with Moderna 100% claimed efficacy against severe disease, for which a more sound 92% should be quoted.

In order to summarize more effectively the probability distribution of $\epsilon$ with just a couple of numbers, our preference goes to mean and standard deviation, although we also report the bounds of the central 95% credible interval. This interval is, once more, in excellent agreement not only with the Pfizer result, which has also published an interval having exactly the same meaning, but also with the uncertainty intervals of the other companies, although they provide confidence intervals, which, strictly speaking, do not have the same meaning of the credible intervals. This is not a surprise to us. We are in fact aware that in many practical cases not only frequentistic point estimates are equivalent to the mode of the posterior distribution of the model parameter, if a uniform prior was used in a Bayesian analysis based on the same data, but also `95% confidence intervals' tend to be, numerically, equal to the 95% probable intervals.²⁰

This takes us to the question of the priors. As just reminded, a uniform prior over $\epsilon$ has been used in our analysis. But, clearly, not because we believe that the efficacy of a vaccine that has reached the Phase-3 trial has the same chance to be close to zero or to one. Instead, a flat prior can be considered a convenient practical choice, if the inference is dominated by the data, as it is often the case. Moreover, the advantage of a uniform prior in parametric inference is that the effect of an informative prior reflecting the opinion of experts can be taken into account at a later time. This `posterior use of priors' might sound paradoxical, but it is important to remind that in Bayesian inference `prior' does not indicate time order but rather `based on the status of knowledge without taking into account the new piece of information' provided by the data entering the specific analysis. Having, in fact, prior and likelihood symmetric and peer roles in the Bayes' rule, an expert can use her prior to `reshape' the posterior pdf resulting from data analysis, if a flat prior was used, without having to ask to repeat the analysis (see Ref. [19] for details).

This reshaping becomes particularly simple if the prior is modeled by a convenient, rather flexible probability distribution such as the Beta. In fact, as we have seen, the pdf of $\epsilon$ starting from a flat prior tends to resemble a Beta. The same is then true if also the prior is modeled by a Beta (this is related to the well known fact of the Beta being the conjugate prior of a binomial distribution, even though our model is not just a simple binomial). These observations are particularly interesting because they lead to Eqs. () - (), which, together with Eqs. () - (), allow to take easily into account the expert priors. In fact, if the priors are rather vague, $r_0$ and $s_0$ appearing in Eqs. () - () are quite small (although larger than one, since $\epsilon=0$ and $\epsilon=1$ are à priori reasonably ruled out) and, in particular, smaller that $r_F$ and $s_F$.²¹If, instead, the expert has a strong opinion about the possible values of $\epsilon$, then $r_0$ and $s_0$ will play a role in her posterior, and in that of her community, if its members trust her.²²

Coming back to the way to summarize $f(\epsilon)$, our preference goes to its mean and standard deviation. The mean because, as reminded above, has the meaning of efficacy for vaccine treated people not having been involved in the trial, if all possible values of $\epsilon$ are taken into account. The standard deviation because it is mostly convenient, together with the mean, to make use of the result of the inference in further considerations and in `propagation of uncertainties', thanks to general probability rules.

We have just reminded the utility of mean and standard deviation in order to re-obtain $f(\epsilon)$, under the hypothesis that it is almost a Beta distribution, making use of Eqs. () - (). The application related to `propagation of uncertainties' that we have seen in the paper has to do with predicting the number of individual that will get infected in a group that it is going to be vaccinated. This is a problem in probabilistic forecasting and the number of interest is uncertain for several reasons. There is, unavoidably, the uncertainty deriving from the inherent binomial distribution, having assumed an assault probability $p_A'$ in the new population. But also the uncertainties on the values of $p_A'$ and $\epsilon$ play a role, that can even be dominant with respect to the `statistical' effect of the binomial.

Now, the probability distribution of the number of vaccinated infectees can be evaluated extending our basic Bayesian network, as we have done here. But we have also stressed the importance of having approximated expressions, based on linearization, for its expected value and standard deviation. And such expressions, thus obtained considering $\epsilon$ and $p_A'$ as `systematic' [16], depend then on their mean and standard deviation. For example the contribution to $\sigma(n_{V_I}')$ due to the uncertain $\epsilon$, and then to be added `in quadrature' to the other sources of uncertainty, is given by ${n'}_V\!\cdot\!$E$(p'_A)\! \cdot\! \sigma(\epsilon)$. This gives at a glance the contribution to the global uncertainty without having to run a Monte Carlo.²³

Finally, a comment on how to possible reduce $\sigma(\epsilon)$ is in order. In fact, the relative uncertainty on $\epsilon$ depends on the small number of vaccinated infectees. This suggests that the quality of its `measurement' could be improved, keeping constant the total numbers of individuals entering the trial, if the size of the placebo group is reduced. We have checked by simulation that reducing it by 2/3, thus having about a factor of five between the two groups, $\sigma(\epsilon)$ is expected to be reduced by about 20%. Not much indeed, but this different sharing of individuals in the two groups would have the advantage of increasing the chance of detecting side effect of the vaccine, basically at the same cost.