Efficacy against disease severity

A last point we wish to address in this paper is related to the efficacy of the vaccines against disease severity, based on the data reported by Moderna and Pfizer (see Tab. ): in the first case 30 people got a `severe form' out of 185 infectees in the control group; none of the severe cases occurred in the group of 11 vaccinated infectees; in the second the corresponding numbers are 9 in 162 and 1 in 8.

In order to analyze this further pieces of information we can simply extend the Bayesian network of Fig.  adding four nodes (see Fig. ):

Figure: Extended Bayesian network of the vaccine vs placebo experiment (see text).

$n_{V_{I_s}}$ and $n_{P_{I_s}}$ represent the number of infected individuals that got the disease in a severe form, while $p_{V_{I_s}}$ and $p_{P_{I_s}}$ represent the corresponding probability of developing severe diseases in either group. Again we can use the binomial distributions:

$$n_{V_{I_s}} \sim (n_{V_I},\, p_{V_{I_s}})$$ (29)

$$n_{P_{I_s}} \sim (n_{P_I},\, p_{P_{I_s}})\,,$$ (30)

and the following JAGS model would result:

    model {
      nP.I  ~ dbin(pA, nP)           #  1.          
      nV.A  ~ dbin(pA, nV)           #  2.
      pA    ~ dbeta(1,1)             #  3. 
      nV.I  ~ dbin(ffe, nV.A)        #  4.  [ ffe = 1 - eff ]
      ffe   ~ dbeta(1,1)             #  5.
      eff   <- 1 - ffe               #  6. 
      pS_P  ~ dbeta(1,1)             #  7.
      pS_V  ~ dbeta(1,1)             #  8.  
      nS.V  ~ dbin(pS_V,nV.I)        #  9.
      nS.P  ~ dbin(pS_P,nP.I)        # 10.
    }

However, looking at the Bayesian network of Fig. , it is clear that, being $n_{V_I}$ and $n_{{P_I}}$ observed nodes, i.e. $n_{V_I}$ and $n_{{P_I}}$ are just data, the bottom nodes involving $(n_{V_{I_s}}, n_{V_I}, p_{V_{I_s}})$ and $(n_{P_{I_s}}, n_{P_I}, p_{P_{I_s}})$ get `separated' from the rest of the network. In other words there is no flow of evidence from $(n_{V_{I_s}}, p_{V_{I_s}})$, or from $(n_{P_{I_s}}, p_{P_{I_s}})$, to the rest of the network. Therefore the problem has a rather simple solution. In particular, using uniform priors for $p_{V_{I_s}}$ and $p_{P_{I_s}}$, we get

$$p_{V_{I_s}} \sim (n_{V_{I_s}}\!+\! 1, n_{V_I}\!-\!n_{V_{I_s}}\!+1)$$

$$p_{P_{I_s}} \sim (n_{P_{I_s}}\!+\! 1, n_{P_I}\!-\!n_{P_{I_s}}\!+1) \,.$$

Nevertheless, for didactic purposes, we report in Fig. 

Figure: Distribution of the probability of getting a severe form of Covid-19 (see text).

the histograms of the MCMC results with superimposed the Beta pdf's (solid lines – we shall come back later to meaning of the dashed line added in the case of Moderna).

As far as the control groups are concerned (green, narrower histograms and curves in Fig. ), the results from Moderna and Pfizer data are quite different. In both cases we get rather narrow distributions, as expected from the rather large numbers involved (and therefore the central values are close to the proportion of severe cases with respect to the total number). But they differ substantially and, using mean and standard deviation to summarize them, we get

$$p_{P_{I_s}}= 0.170 \pm 0.028$$ Moderna:

$$% dalle Beta !! p_{P_{I_s}}=0.055 \pm 0.018\,,$$

differing by $0.115\pm 0.033$. In particular, what results by analyzing Moderna data is in good agreement with the estimates by the WHO, which states that “approximately 10-15% of cases progress to severe disease, and about 5% become critically ill” [26]. Obviously, we are not in the position to make any statement about the reason of the disagreement, that could be due to the different populations on which the trials have been performed. But, if this were the case, one could have some doubts about the validity of comparing the different sets of data. We can only leave the question to epidemiology experts.

Passing to the vaccine groups (red, broader histograms and curves in Fig. ), the crude summaries in terms of mean and standard deviation give

$$p_{V_{I_s}}=0.077 \pm 0.071$$ Moderna:

$$p_{V_{I_s}}=0.200 \pm 0.121\,.$$ Pfizer:

If we consider just the mean values, it seems that there is a very large difference between the performances of the two vaccines. However their difference is hardly significant, being their difference $-0.12 \pm 0.14$. Also the fact that the two curves looks substantially different should not impress a statistics expert if she knew from which number they have been derived. The main difference is due to the fact that Moderna has zero severe disease against one of Pfizer. If they had had one too, which would not be surprising in a hypothetical `second trial', the curve would be not dissimilar from the Pfizer one (dashed line in Fig. ), vanishing at zero and yielding $p_{V_{I_s}}=0.154 \pm 0.096$ – practically the same, an experienced physicist would say.

It is quite evident that it is not possible to draw general conclusions on the efficacy of the generic vaccine on softening the impact of the disease. But the real point we wish to highlight, given the spread of distributions, is that we do not have enough data for drawing sound conclusion. For this reason we wish to point out that, even for this aspect, press releasing a 100% efficacy and not dealing with the unavoidable uncertainties and their impact when applied to decision making is quite misleading. Figure indeed shows that the probability of becoming severely ill in the vaccine group is definitively low but, quite obviously, not zero and with a relevant overlap with the distribution evaluated for the control group.