Hit and run MCMC

I’m trying to implement the hit and run MCMC algorithm, but I’m having a bit of trouble understanding how to go about it. The general idea, is as follows:

To generate a proposal jump in MH, we:

  • Generate a direction $d$ from a
    distribution on the surface of the
    unit sphere $\mathcal{O}$
  • Generate
    a signed distance $\lambda$ along the
    constrained space.

However, I have no idea of how I should go about implementing this in R (or any other language).

Does anyone have a snippet of code that would point me in the right direction?

BTW, I not that interested in a library that does this method, I want to try and code it up myself.

Many thanks.

Answer

I didn’t look at the paper you supplied, but let me have a go anyway:

If you have a $p$-dimensional parameter space you can generate a random direction $d$ uniformly distributed on the surface of the unit sphere with

x <- rnorm(p)
d <- x/sqrt(sum(x^2))

(c.f. Wiki).

Then, use this to generate proposals for $d$ for rejection sampling (assuming you can actually evaluate the distribution for $d$).

Assuming you start in position $x$ and have accepted a $d$, generate a proposal $y$ with

 lambda <- r<SOMEDISTRIBUTION>(foo, bar)
 y <- x + lambda * d

and do a Metropolis-Hastings-Step to decide whether to move to $y$ or not.

Of course, how well this can work will depend on the distribution of $d$ and how expensive it is to (repeatedly) evaluate its density in the rejection sampling step, but since generating proposals for $d$ is cheap you may get away with it.


Added for @csgillespie’s benefit:

From what I was able to gather by some googling, hit-and-run MCMC is useful primarily for fast mixing if you have a (multivariate) target that has arbitrary bounded but not necessarily connected support, because it enables you to move from any point in the support to any other in one step. More here and here.

Attribution
Source : Link , Question Author : Community , Answer Author : BonStats

Leave a Comment