Distribution of several random variables

We only consider the case of two continuous variables ($X$ and $Y$). The extension to more variables is straightforward. The infinitesimal element of probability is $\rm {d}F(x,y) = f(x,y)\,\rm {d}x\,\rm {d}y$, and the probability density function

$$f(x,y) = \frac{\partial^2F(x,y)}{\partial x\partial y}\,.$$ (4.43)

The probability of finding the variable inside a certain area $A$ is

$$\iint\limits_A f(x,y)\,\rm {d}x\,\rm {d}y\,.$$ (4.44)

Marginal distributions:

$$f_X(x) = \int_{-\infty}^{+\infty}f(x,y)\,\rm {d}y,$$ (4.45)

$$f_Y(y) = \int_{-\infty}^{+\infty}f(x,y)\,\rm {d}x \,.$$ (4.46)

The subscripts $X$ and $Y$ indicate that $f_X(x)$ and $f_Y(y)$ are only functions of $X$ and $Y$, respectively (to avoid fooling around with different symbols to indicate the generic function), but in most cases we will drop the subscripts if the context helps in resolving ambiguities.

Conditional distributions:

$$f_X(x\,\vert\,y) = \frac{f(x,y)}{f_Y(y)} = \frac{f(x,y)}{\int f(x,y)\,\rm {d}x},$$ (4.47)

$$f_Y(y\,\vert\,x) = \frac{f(x,y)}{f_X(x)},$$ (4.48)

$$f(x,y) = f_X(x\,\vert\,y)\,f_Y(y)$$ (4.49)

$$= f_Y(y\,\vert\,x)\,f_X(x)\,.$$ (4.50)

Independent random variables

$$f(x,y) = f_X(x) f_Y(y)$$ (4.51)

(it implies $f_X(x\,\vert\,y)=f_X(x)$ and $f_Y(y\,\vert\,x)=f_Y(y)$.)

Bayes' theorem for continuous random variables

$$\boxed{ f(h\,\vert\,e) = \frac{f(e\,\vert\,h)\,f_h(h)} {\int f(e\,\vert\,h)\,f_h(h)\,\rm{d}h}\, . }$$ (4.52)

(Note added: see proof in section .)

Expectation value:

$$\mu_X= [X] = \int\!\!\int_{-\infty}^{+\infty} x\, f(x,y)\,\rm {d}x\,\rm {d}y$$ (4.53)

$$= \int_{-\infty}^{+\infty}\!x\, f_X(x)\, \rm {d}x\,,$$ (4.54)

and analogously for $Y$. In general

$$[g(X,Y)] = \int\!\!\int_{-\infty}^{+\infty} \!g(x,y)\, f(x,y)\, \rm {d}x\,\rm {d}y\,.$$ (4.55)

Variance:

$$\sigma_X^2= [X^2]- ^2[X]\,,$$ (4.56)

and analogously for $Y$.

Covariance:

$$(X,Y) = \left[\,\left(X-\mbox{E}[X]\right)\cdot \left(Y-\mbox{E}[Y]\right)\,\right]$$ (4.57)

$$= [X Y]- [X]\cdot [Y]\,.$$ (4.58)

If $X$ and $Y$ are independent, then    E$[XY]=$E$[X]\cdot$   E$[Y]$ and hence Cov$(X,Y) =0$ (the opposite is true only if $X$, $Y\sim {\cal N}(\cdot)$).

Correlation coefficient:

$$\rho(X,Y) = \frac{\mbox{Cov}(X,Y)}{\sqrt{\mbox{Var}(X)\, \mbox{Var}(Y)}}$$ (4.59)

$$= \frac{\mbox{Cov}(X,Y)}{\sigma_X\, \sigma_Y}\, .$$ (4.60)

$$( -1 \le \rho \le 1)$$

Linear combinations of random variables:

If $Y=\sum_i c_iX_i$, with $c_i$ real, then:

$$\mu_Y= [Y] = \sum_ic_i\, [X_i] = \sum_ic_i\,\mu_i,$$ (4.61)

$$\sigma_Y^2= (Y) = \sum_i c_i^2\, (X_i) + 2\sum_{i< j}c_i\,c_j\, (X_i,X_j)$$ (4.62)

$$= \sum_i c_i^2\, (X_i) + \sum_{i\ne j}c_i\,c_j\, (X_i,X_j)$$ (4.63)

$$= \sum_i c_i^2\,\sigma_i^2 +\sum_{i\ne j}\rho_{ij}\,c_i\,c_j\,\sigma_i\,\sigma_j$$ (4.64)

$$= \sum_{ij}\rho_{ij}\,c_i\,c_j\,\sigma_i\,\sigma_j$$ (4.65)

$$= \sum_{ij}c_i\,c_j\,\sigma_{ij}\,.$$ (4.66)

$\sigma^2_Y$ has been written in different ways, with increasing levels of compactness, that can be found in the literature. In particular, () and () use the notations $\sigma_{ij}\equiv$   Cov$(X_i,X_j) = \rho_{ij}\,\sigma_i\,\sigma_j$ and $\sigma_{ii}=\sigma^2_i$, and the fact that, by definition, $\rho_{ii}=1$.

Bivariate normal distribution:

Joint probability density function of $X$ and $Y$ with correlation coefficient $\rho$ (see Fig. ):

Figure: Example of bivariate normal distribution.

$$f(x,y) = \frac{1}{2\,\pi\,\sigma_x\,\sigma_y\,\sqrt{1-\rho^2}}\cdot$$ (4.67)

$$\exp{\left\{ -\frac{1}{2\,(1-\rho^2)} \left[ \frac{(x-\mu_x)^2}{\... ...u_y)}{\sigma_x\,\sigma_y} + \frac{(y-\mu_y)^2}{\sigma_y^2} \right] \right\}}\,.$$

Marginal distributions:

$$X \sim {\cal N}(\mu_x,\sigma_x),$$ (4.68)

$$Y \sim {\cal N}(\mu_y,\sigma_y) \,.$$ (4.69)

Conditional distribution:

$$f(y\,\vert\,x_\circ) = \frac{1}{\sqrt{2\,\pi}\,\sigma_y\,\sqrt{1-... ...(x_\circ-\mu_x\right)\right] \right)^2} {2\,\sigma_y^2\,(1-\rho^2)} \right]}\,,$$ (4.70)

i.e.

$$Y_{\vert x_\circ}\sim {\cal N}\left( \mu_y+\rho\,\frac{\sigma_y}{\sigma_x} \,\left(x_\circ-\mu_x\right),\, \sigma_y\sqrt{1-\rho^2}\right).$$ (4.71)

The condition $X=x_\circ$ squeezes the standard deviation and shifts the mean of $Y$.