Covariance matrix proposal distribution

In a MCMC implementation of hierarchical models, with normal random effects and a Wishart prior for their covariance matrix, Gibbs sampling is typically used.

However, if we change the distribution of the random effects (e.g., to Student’s-t or another one), the conjugacy is lost. In this case, what would be a suitable (i.e., easily tunable) proposal distribution for the covariance matrix of the random effects in a Metropolis-Hastings algorithm, and what should be the target acceptance rate, again 0.234?

Thanks in advance for any pointers.

Answer

Well, if you are looking “for any pointers”…

The (scaled)(inverse)Wishart distribution is often used because it is conjugate
to the multivariate likelihood function and thus simplifies Gibbs sampling.

In Stan, which uses Hamiltonian Monte Carlo sampling, there is no restriction for multivariate priors. The recommended approach is the separation strategy suggested by Barnard, McCulloch and Meng:
Σ=diag_matrix(σ)Ωdiag_matrix(σ)
where σ is a vector of std devs and Ω is a correlation matrix.

The components of σ can be given any reasonable prior. As to Ω, the recommended prior is
ΩLKJcorr(ν)
where “LKJ” means Lewandowski, Kurowicka and Joe. As ν increases, the prior increasingly concentrates around the unit correlation matrix, at ν=1 the LKJ correlation distribution reduces to the identity distribution over correlation matrices. The LKJ prior may thus be used to control the expected amount of correlation among the parameters.

However, I’ve not (yet) tried non-normal distributions of random effects, so I hope I’ve not missed the point 😉

Attribution
Source : Link , Question Author : Toka Stall , Answer Author : Sergio

Leave a Comment