Reversible jump MCMC

Reversible jump MCMC

Reversible jump MCMC is a Bayesian algorithm to infer the number of components/ clusters from a set of data. For this illustration we shall consider a two component model at most.

Model

The likelihoods can be represented as:

The priors on the latent variables are:

Jumping dimensions

We need to consider a Metropolis-Hastings (MH) step to consider going from one component to two components. The MH step in general is as follows:

where,

Jumping from 1 dim to 2

In this case let the parameters . As we can let the proposal distribution be anything, we let as follows:

We let the proposal . We also have the following dimensional jump:

Thus, in order to find the distribution (q(\lambda_{j2}\vert k=2,\lambda_{11})) we use the change of variable identity that where, is the jacobian . The Jacobian determinant is found to be while since they are sampled from standard uniform distributions. Also (q(\mu_2)=q(\pi\vert k=2)).

Since we need the ratio of proposed states ( \frac{q(\theta_1^t \theta_2^{t+1})}{q(\theta_2^{t+1} \theta_1^{t})} ) we are also required to find ( q(\lambda_{11},k=2\vert\lambda_{2j},\pi,k=1) = q(\lambda_{11}\vert\lambda_{2j},k=2) q(k=1 \vert k=2) ). We again take ( q(k=1\vert k=2)=1 ). (q(\lambda_{11}=\sqrt{\lambda_{12}\lambda_{22}})=1)

Jumping from 2 to 1

The MH step is conducted using the reciprocal of $\alpha$ in the equation above.

RJMCMC Algorithm

comments powered by Disqus