What is a ‘message passing method’?

I have a vague sense of what a message passing method is: an algorithm that builds an approximation to a distribution by iteratively building approximations of each of the factors of the distribution conditional on all the approximations of all the other factors.

I believe that both are examples Variational Message Passing and Expectation Propagation. What is a message passing algorithm more explicitly/correctly? References are welcome.

Answer

Since you ask for references, I can recommend chapter 16 of David MacKay’s Information Theory, Inference, and Learning Algorithms. (you don’t need to read the previous 15 chapters to understand ch. 16) The book is free for downloading from the author’s website (with permission from the publisher).

For an interesting example, check out the thesis of John Winn. Uses a message passing algorithm for generic Variational Ensemble Learning – enabling simple construction of inference problems such as ICA and PCA.

Attribution
Source : Link , Question Author : John Salvatier , Answer Author : Tom

Leave a Comment