Gaussian approximation of the posterior distribution

The substance of the results seen in the previous section holds also in the case in which the prior is not flat and, hence, cannot be absorbed in the normalization constant of the posterior. In fact, in many practical cases the posterior exhibits an approximately (multi-variate) Gaussian shape, even if the prior was not trivial. Having at hand an un-normalized posterior $\tilde p()$, i.e.

$$\tilde p({\mbox{\boldmath$\theta$}} \,\vert\,{\mbox{\boldmath$d$}},I) = p({\mbox{\boldmath$d$}} \,\vert\,{\mbox{\boldmath$\theta$}},I) \, p_0({\mbox{\boldmath$\theta$}},I)\,,$$ (69)

we can take its minus-log function $\varphi({\mbox{\boldmath$\theta$}} ) = -\ln \tilde p({\mbox{\boldmath$\theta$}})$. If $\tilde p({\mbox{\boldmath$\theta$}} \,\vert\,{\mbox{\boldmath$x$}},{\mbox{\boldmath$y$}},I)$ has approximately a Gaussian shape, it follows that

$$\varphi({\mbox{\boldmath$\theta$}} ) \approx \frac{1}{2}\,{\mbox{\boldmath$\Delta$}}\theta^T \, \mathbf{V}^{-1}{\mbox{\boldmath$\Delta$}}\theta + \mbox{\it constant}\,.$$ (70)

$\mathbf{V}$ can be evaluated as

$$(V^{-1})_{ij} ({\mbox{\boldmath$\theta$}}) \approx \left. \frac{\partial^2\varphi}{\partial\theta_i\partial\theta_j} \right\vert _{{\mbox{\boldmath$\theta$}}={\mbox{\boldmath$\theta$}}_m}\,,$$ (71)

where ${\mbox{\boldmath$\theta$}}_m$ was obtained from the minimum of $\varphi({\mbox{\boldmath$\theta$}})$.