# Statistical learning when observations are not iid

As far as I am concerned, statistical/machine learning algorithms always suppose that data are independent and identically distributed ($$iidiid$$).

My question is: what can we do when this assumption is clearly unsatisfied? For instance, suppose that we have a data set whith repeated measurements on the same observations , so that both the cross-section and the time dimensions are important (what econometricians call a panel data set, or statisticians refer to as longitudinal data, which is distinct from a time series).

An example could be the following. In 2002, we collect the prices (henceforth $$YY$$) of 1000 houses in New York, together with a set of covariates (henceforth $$XX$$). In 2005, we collect the same variables on the same houses. Similar happens in 2009 and 2012. Say I want to understand the relationship between $$XX$$ and $$YY$$. Were the data $$iidiid$$, I could easily fit a random forest (or any other supervised algorithm, for what matters), thus estimating the conditional expectation of $$YY$$ given $$XX$$. However, there is clearly some auto-correlation in my data. How can I handle this?

When samples are i.i.d, you can write the joint probability of the samples given some model as a product, namely $$P({x})=ΠiPi(xi)P(\{x\}) = \Pi_{i} P_i(x_i)$$ which makes the log-likelihood a sum of the individual log-likelihoods. This simplifies the calculation, but is by no means a requirement.
In your case, you can for example model the distribution of a pair $$xi,yix_i,y_i$$ with some bi-variate distribution, say $$zi=(xi,yi)Tz_i=(x_i,y_i)^T$$ , $$zi∼N(μ,Σ)z_i \sim \mathcal{N}(\mu,\Sigma)$$ , and then estimate the parameter $$Σ\Sigma$$ from the likelihood $$P({z})=ΠiP(zi|μ,Σ)P(\{z\}) = \Pi_{i} P(z_i | \mu, \Sigma)$$.