Due to the linearity property of the expectations, we can write E(X+Y)=E(X)+E(Y). But in Chapter 6.1 of the book, “Counterexamples in probability“, the writer, Jordan Stoyanov argued that this is not true for three random variables, i.e., for E(X+Y+Z).
The paper cited as the reference in that chapter is “An Unexpected Expectation“(1977) by Gordon Simons. But unfortunately, in that paper, the writer didn’t provide any formal proof for this phenomenon. And stated that “We have no explanations why the theorem given above should hold for two variables but fail for three.”
I haven’t quite understood why this is the case or found any proof for this. Can anyone please help me understand why the linearity property will not be held for the case of three random variables? And please correct me if I am getting it totally wrong.
Simons’ counterexample considers random variables that don’t have finite marginal expectations. If you sum 3 (or more) random variables that have finite marginal expectations, then the expectation of their sum equals the sum of their expectations. So to say “this is not true for three random variables” is not quite right.
Based on a comment from the OP, the content below is edited from my original response.
What I interpret Simons to be claiming is that if E[X+Y]+ or E[X+Y]− is finite, then to calculate E[X+Y] you do not need to know or assume anything about how X and Y are jointly distributed. Rather, you only need to know about marginal distributions. But, he says, this doesn’t extend to the case of three or more variables. In other words, the finitude of E[X+Y+Z]+ or E[X+Y+Z]− does not imply that one can calculate E[X+Y+Z] using only knowledge of the marginal distributions of X, Y, and Z. He gives a counterexample to demonstrate this point.
The result only seems to contradict the usual thing we teach (“expectations are additive”) because we are implicitly assuming that the marginal expectations are finite. Simons is allowing for non-finite marginal expectations.
When Simon writes We have no explanations why the theorem given above should hold for two variables but fail for three, I understand this to be him saying that he does have a good intuitive explanation for why this mathematical result – which he has shown rigorously – holds.