Including systematic effects

A last interesting case is when there are systematic errors of unknown size in the detector performance. Independently of where systematic errors may enter, the final result will be an uncertainty on ${\cal L}$. In the most general case, the uncertainty can be described by a probability density function:

$$f({\cal L}) = f({\cal L}\,\vert\,$$best knowledge on experiment$$)\,.$$

For simplicity we analyse here only the case of a single experiment. In the case of many experiments, we only need to iterate the Bayesian inference, as has often been shown in these notes.

Following the general lines given in Section , the problem can be solved by considering the conditional probability, obtaining :

$$f(r\,\vert\, ) = \int f(r\,\vert\, , {\cal L})\, f({\cal L})\, {\cal L}\,.$$ (9.11)

The case of absolutely precise knowledge of ${\cal L}$ is recovered when $f({\cal L})$ is a Dirac delta.

Let us treat in some more detail the case of null observation ( $\underline{x}=\underline{0}$). For each possible value of ${\cal L}$ one has an exponential of expected value $1/{\cal L}$ [see Eq. ()]. Each of the exponentials is weighted with $f({\cal L})$. This means that, if $f({\cal L})$ is rather symmetrical around its barycentre (expected value), in a first approximation the more and less steep exponentials will compensate, and the result of integral () will be close to $f(r)$ calculated in the barycentre of ${\cal L}$, i.e. in its nominal value ${\cal L}_\circ$:

$$f(r\,\vert\, ) = \int f(r\,\vert\, , {\cal L})\, f({\cal L})\, {\cal L}\, \approx f(r\,\vert\, , {\cal L}_\circ)$$

$$r_u \,\vert\, \approx r_u \,\vert\, ,{\cal L}_\circ\,.$$

Figure: Inference on the rate of a process, with and without taking into account systematic effects: upper plot: difference between $f(r\,\vert\,x=0,{\cal L}=1.0\pm 0.1)$ and $f(r\,\vert\,x=0,{\cal L}=1\pm 0)$, using a normal distribution of ${\cal L}$; lower plot: integral of the difference, to give a direct idea of the variation of the upper limit.

To make a numerical example, let us consider ${\cal L}=1.0\pm 0.1$ (arbitrary units), with $f({\cal L})$ following a normal distribution. The upper plot of Fig. shows the difference between $f(r\,\vert\,$Data$)$ calculated applying Eq. () and the result obtained with the nominal value ${\cal L}_\circ = 1$:

$$df = f(r\,\vert\,x=0, f({\cal L})\,) - f(r\,\vert\,x=0, {\cal L}=1.0)$$ (9.12)

$$= \int f(r\,\vert\,x=0, {\cal L})\, f({\cal L})\, {\cal L} - e^{-r} \,.$$ (9.13)

$d$ is negative up to $r\approx 2$, indicating that systematic errors normally distributed tend to increase the upper limit. But the size of the effect is very tiny, and depends on the probability level chosen for the upper limit. This can be seen better in the lower plot of Fig. , which shows the integral of the difference of the two functions. The maximum difference is for $r\approx 2$. As far as the upper limits are concerned, we obtain (the large number of -- non-significatant--digits is only to observe the behaviour in detail):

$$r_u (x=0, {\cal L} =1 \pm 0, \, 90\%) = 2.304$$

$$r_u (x=0, {\cal L} =1.0\pm 0.1, \, 90\%) = 2.330$$

$$r_u (x=0, {\cal L} =1 \pm 0, \, 95\%) = 2.996$$

$$r_u (x=0, {\cal L} =1.0\pm 0.1, \, 95\%) = 3.042\, .$$

An uncertainty of 10% due to systematics produces only a 1% variation of the limits.

To simplify the calculation (and also to get a feeling of what is going on) we can use some approximations.

1. Since the dependence of the upper limit of $r$ from $1/{\cal L}$ is given by
   $$r_u = \frac{-\ln(1-P_u)}{{\cal L}}\,,$$
   the upper limit averaged with the belief on ${\cal L}$ is given by
   $$r_u = -\ln(1-P_u)\,$$   E$$\left[\frac{1}{{\cal L}}\right] = \int \frac{1}{{\cal L}}f({\cal L})\,$$d$${\cal L}\,.$$
   We need to solve an integral simpler than in the previous case. For the above example of ${\cal L}=1.0\pm 0.1$ we obtain $r_u= 2.326$ at 90% and $r_u= 3.026$ at 95%.
2. Finally, as a real rough approximation, we can take into account the small asymmetry of $r_u$ around the value obtained at the nominal value of ${\cal L}$ averaging the two values of ${\cal L}$ at $\pm\sigma_{\cal L}$ from ${\cal L}_\circ$:
   

   $$r_u \approx \frac{-\ln(1-P_u)}{2} \left(\frac{1}{{\cal L}_\circ-\sigma_{\cal L}} + \frac{1}{{\cal L}_\circ+\sigma_{\cal L}}\right)$$

   $$\approx \frac{-\ln(1-P_u)} {{\cal L}_\circ} \left( 1+\frac{\sigma_{\cal L}^2} {{\cal L}_\circ^2} \right)\,.$$

   
   We obtain numerically identical results to the previous approximation.

The main conclusion is that the uncertainty due to systematics plays only a second-order role, and it can be neglected for all practical purposes. A second observation is that this uncertainty increases slightly the limits if $f({\cal L})$ is distributed normally, but the effect could also be negative if the $f({\cal L})$ is asymmetric with positive skewness.

As a more general remark, one should not forget that the upper limit has the meaning of an uncertainty and not of a value of quantity. Therefore, as nobody really cares about an uncertainty of 10 or 20% on the uncertainty, the same is true for upper/lower limits. At the per cent level it is mere numerology (I have calculated it at the $10^{-4}$ level just for mathematical curiosity).