What is the connection between Markov chain and Markov chain monte carlo

I am trying to understand Markov chains using SAS. I understand that a Markov process is one where the future state depends only on the current state and not on the past state and there is a transition matrix that captures the transition probability from one state to another.

But then I came across this term :Markov Chain Monte Carlo. What I want to know is if Markov Chain Monte Carlo is in anyway related to Markov process that I describe above?

Answer

Well, yes, there is a relationship between the two terms because the draws from MCMC form a Markov chain. From Gelman, Bayesian Data Analysis (3rd ed), p. 265:

Markov chain simulation (also called Markov chain Monte Carlo or MCMC) is a general method based on drawing values of $\theta$ from appropriate distributions and then correcting those draws to better approximate the target posterior distribution, $p(\theta|y)$. The sampling is done sequentially, with the distribution of the sampled draws depending on the last value drawn; hence, the draws form a Markov chain.

Attribution
Source : Link , Question Author : Victor , Answer Author : Cliff AB

Leave a Comment