What is the major difference between correlation and mutual information?

my question particularly applies to network reconstruction

Answer

Correlation measures the linear relationship (Pearson’s correlation) or monotonic relationship (Spearman’s correlation) between two variables, X and Y.

Mutual information is more general and measures the reduction of uncertainty in Y after observing X. It is the KL distance between the joint density and the product of the individual densities. So MI can measure non-monotonic relationships and other more complicated relationships.

Attribution
Source : Link , Question Author : Community , Answer Author : Rob Hyndman

Leave a Comment