# Reversible jump MCMC

Reversible jump MCMC is a Bayesian algorithm to infer the number of components/ clusters from a set of data. For this illustration we shall consider a two component model at most.

## Model

The likelihoods can be represented as: %

The priors on the latent variables are:

## Jumping dimensions

We need to consider a Metropolis-Hastings (MH) step to consider going from one component to two components. The MH step in general is as follows:

where,

### Jumping from 1 dim to 2

In this case let the parameters $\theta=\{\cup_j\lambda_{jk},k,\pi\}$ . As we can let the proposal distribution be anything, we let $q(\theta_1\to\theta_2)$ as follows: \begin{align} q(\lambda_{j2},\pi,k=2|k=1,\lambda_{11})=q(\lambda_{j2}|k=2,\lambda_{11})q(\pi|k=2)q(k=2|k=1) \end{align}

We let the proposal $q(k=2\vert k=1)=1$. We also have the following dimensional jump:

Thus, in order to find the distribution (q(\lambda_{j2}\vert k=2,\lambda_{11})) we use the change of variable identity that $q(\lambda_{j2}\vert k=2,\lambda_{11})=q(\mu_1)\vert J\vert$ where, $J$ is the jacobian $\frac{\partial(\lambda_{11},\mu_1)}{\partial(\lambda_{12},\lambda_{22})}$. The Jacobian determinant is found to be $\frac{\mu_1(1-\mu_1)}{2\lambda_{11}}$ while $q(\mu_1)=q(\mu_2)=1$ since they are sampled from standard uniform distributions. Also (q(\mu_2)=q(\pi\vert k=2)).

 Since we need the ratio of proposed states ( \frac{q(\theta_1^t \theta_2^{t+1})}{q(\theta_2^{t+1} \theta_1^{t})} ) we are also required to find ( q(\lambda_{11},k=2\vert\lambda_{2j},\pi,k=1) = q(\lambda_{11}\vert\lambda_{2j},k=2) q(k=1 \vert k=2) ). We again take ( q(k=1\vert k=2)=1 ). (q(\lambda_{11}=\sqrt{\lambda_{12}\lambda_{22}})=1)

### Jumping from 2 to 1

The MH step is conducted using the reciprocal of $\alpha$ in the equation above.