Can somebody illustrate, as Greg does, but in more detail, how random variables can be dependent, but have zero covariance? Greg, a poster here, gives an example using a circle here.
Can somebody explain this process in more detail using a sequence of steps that illustrate the process at several stages?
Also, if you know of an example from psychology, please illustrate with this concept with a related example. Please be very precise and sequential in your explanation, and also state what some of the consequences might be.
The basic idea here is that covariance only measures one particular type of dependence, therefore the two are not equivalent. Specifically,
Covariance is a measure how linearly related two variables are. If two variables are non-linearly related, this will not be reflected in the covariance. A more detailed description can be found here.
Dependence between random variables refers to any type of relationship between the two that causes them to act differently “together” than they do “by themselves”. Specifically, dependence between random variables subsumes any relationship between the two that causes their joint distribution to not be the product of their marginal distributions. This includes linear relationships as well as many many others.
If two variables are non-linearly related, then they can potentially have 0 covariance but are still dependent – many examples are given here and this plot below from wikipedia gives some graphical examples in the bottom row:
One example where zero covariance and independence between random variables are equivalent conditions is when the variables are jointly normally distributed (that is, the two variables follow a bivariate normal distribution, which is not equivalent to the two variables being individually normally distributed). Another special case is that pairs of bernoulli variables are uncorrelated if and only if they are independent (thanks @cardinal). But, in general the two cannot be taken to be equivalent.
Therefore, one cannot, in general, conclude that two variables are independent just because they appear uncorrelated (e.g. didn’t fail to reject the null hypothesis of no correlation). One is well advised to plot data to infer whether the two are related, not just stopping at a test of correlation. For example, (thanks @gung), if one were to run a linear regression (i.e. testing for non-zero correlation) and found a non-sigificant result, one may be tempted to conclude that the variables are not related, but you’ve only investigated a linear relationship.
I don’t know much about Psychology but it makes sense that there could be non-linear relationships between variables there. As a toy example, it seems possible that cognitive ability is non-linearly related to age – very young and very old people are not as sharp as a 30 year old. If one were to plot some measure of cognitive ablity vs. age one may expect to see that cognitive ability is highest at a moderate age and decays around that, which would be a non-linear pattern.