Linear regression, conditional expectations and expected values

Okay so just a bit hazy on a few things, any help would be much appreciated.
It is my understanding that the linear regression model is predicted via a conditional expectation

E(Y|X)=b+Xb+e

  1. Do we assume that both X and Y are random variables with some unknown probability distribution? it was my understanding that only the residuals and the estimated beta coefficients were random variables. if so, as an example, if Y= obesity and X= age, if we take the conditional expectation E(Y|X=35) meaning, whats the expected value of being obese if the individual is 35 across the sample, would we just take the average(arithmetic mean) of y for those observations where X=35? yet doesn’t the expected value entail that we must multiply this by the probability of occurring ? but how in that sense to we find the probability of the X-value variable occurring if it represent something like age?
  2. If X represented something like the exchange rate, would this be classified as random? how on earth would you find the expected value of this without knowing the probability though? or would the expected value just equal the mean in the limit.
  3. If we don’t assume the dependent variables are themselves random variables, since we don’t obverse the probability, what do we assume they are? just fixed values or something? but if this is the case, how can we condition on a non-random variable to begin with? what do we assume about the independent variables distribution?

Sorry if anything doesn’t make sense or is obvious to anyone.

Answer

In the probability model underlying linear regression, X and Y are random variables.

if so, as an example, if Y = obesity and X = age, if we take the conditional expectation E(Y|X=35) meaning, whats the expected value of being obese if the individual is 35 across the sample, would we just take the average(arithmetic mean) of y for those observations where X=35?

That’s right. In general, you cannot expect that you will have enough data at each specific value of X, or it may be impossible to do so if X can take a continuous range of values. But conceptually, this is correct.

yet doesn’t the expected value entail that we must multiply this by the probability of occurring ?

This is the difference between the unconditional expectation E[Y] and the conditional expectation E[YX=x]. The relationship between them is

E[Y]=xE[YX=x]Pr[X=x]

which is the law of total expectation.

but how in that sense to we find the probability of the X-value variable occurring if it represent something like age?

Generally you don’t in linear regression. Since we are attempting to determine E[YX], we don’t need to know Pr[X=x].

If we don’t assume the independent variables are themselves random variables, since we don’t obverse the probability, what do we assume they are? just fixed values or something?

We do assume that Y is a random variable. One way to think about linear regression is as a probability model for Y

YXβ+N(0,σ)

Which says that, once you know the value of X, the random variation in Y is confined to the summand N(0,σ).

Attribution
Source : Link , Question Author : William Carulli , Answer Author : Matthew Drury

Leave a Comment