Transformation invariance

An important class of priors arises from the requirement of transformation invariance. We shall consider two specific cases, translation invariance and scale invariance.

Translation invariance

Let us assume we are indifferent over a transformation of the kind $\theta' = \theta\,+\,b$, where $\theta$ is our variable of interest and $b$ a constant. Then $p(\theta)\,\,\mbox{d}\theta$ is an infinitesimal mass element of probability for $\theta$ to be in the interval $\,\mbox{d}\theta$. Translation invariance requires that this mass element remains unchanged when expressed in terms of $\theta'$, i.e.

$$p(\theta) \, \,\mbox{d}\theta = p(\theta') \, \,\mbox{d}\theta'$$ (100)

$$= p(\theta + b) \, \,\mbox{d}\theta \,,$$ (101)

since $\,\mbox{d}\theta = \,\mbox{d}\theta'$. It is easy to see that in order for Eq. (101) to hold for any $b$, $p(\theta)$ must be equal to a constant for all values of $\theta$ from $-\infty$ to $+\infty$. It is therefore an improper prior. As discussed above, this is just a convenient modelling. For practical purposes this prior should always be regarded as the limit for $\Delta\theta\rightarrow \infty$ of $p(\theta) = 1/\Delta \theta$, where $\Delta \theta$ is a large finite range around the values of interest.

Scale invariance

In other cases, we could be indifferent about a scale transformation, that is $\theta' = \beta \,\theta$, where $\beta$ is a constant. This invariance implies, since $\,\mbox{d}\theta' = \beta \,\mbox{d}\theta$ in this case,

$$p(\theta) \, \,\mbox{d}\theta = p(\beta\, \theta) \, \beta \,\mbox{d}\theta \,,$$ (102)

i.e.

$$p(\beta\, \theta) = \frac{p(\theta)}{\beta}\,.$$ (103)

The solution of this functional equation is

$$p(\theta) \propto \frac{1}{\theta}\,,$$ (104)

as can be easily proved using Eq. (104) as test solution in Eq. (103). This is the famous Jeffreys' prior, since it was first proposed by Jeffreys. Note that this prior also can be stated as $p(\log \theta) = \mbox{constant}$, as can be easily verified. The requirement of scale invariance also produces an improper prior, in the range $0 < \theta <\infty$. Again, the improper prior must be understood as the limit of a proper prior extending several orders of magnitude around the values of interest. [Note that we constrain $\theta$ to be positive because, traditionally, variables which are believed to satisfy this invariance are associated with positively defined quantities. Indeed, Eq. (104) has a symmetric solution for negative quantities.]

According to the supporters of these invariance motivated priors (see e.g. Jaynes 1968, 1973, 1998, Sivia 1997, and Fröhner 2000, Dose 2002) variables associated to translation invariance are location parameters, as the parameter $\mu$ in a Gaussian model; variables associated to scale invariance are scale parameters, like $\sigma$ in a Gaussian model or $\lambda$ in a Poisson model. For criticism about the (mis-)use of this kind of prior see (D'Agostini 1999d).