What is an example of perfect collinearity in terms of the design matrix X?

I would like an example where ˆβ=(X′X)−1X′Y can’t be estimated because (X′X) is not invertible.

**Answer**

Here is an example with 3 variables, y, x1 and x2, related by the equation

y=x1+x2+ε

where ε∼N(0,1)

The particular data are

```
y x1 x2
1 4.520866 1 2
2 6.849811 2 4
3 6.539804 3 6
```

So it is evident that x2 is a multiple of x1 hence we have perfect collinearity.

We can write the model as

Y=Xβ+ε

where:

Y=[4.526.856.54]

X=[112124136]

So we have

XX′=[112124136][111123246]=[61116112131163146]

Now we calculate the determinant of XX′ :

det

In R we can show this as follows:

```
> x1 <- c(1,2,3)
```

create `x2`

, a multiple of `x1`

```
> x2 <- x1*2
```

create y, a linear combination of `x1`

, `x2`

and some randomness

```
> y <- x1 + x2 + rnorm(3,0,1)
```

observe that

```
> summary(m0 <- lm(y~x1+x2))
```

fails to estimate a value for the `x2`

coefficient:

```
Coefficients: (1 not defined because of singularities)
Estimate Std. Error t value Pr(>|t|)
(Intercept) 3.9512 1.6457 2.401 0.251
x1 1.0095 0.7618 1.325 0.412
x2 NA NA NA NA
Residual standard error: 0.02583 on 1 degrees of freedom
Multiple R-squared: 1, Adjusted R-squared: 0.9999
F-statistic: 2.981e+04 on 1 and 1 DF, p-value: 0.003687
```

The model matrix X is:

```
> (X <- model.matrix(m0))
(Intercept) x1 x2
1 1 1 2
2 1 2 4
3 1 3 6
```

So XX’ is

```
> (XXdash <- X %*% t(X))
1 2 3
1 6 11 16
2 11 21 31
3 16 31 46
```

which is not invertible, as shown by

```
> solve(XXdash)
Error in solve.default(XXdash) :
Lapack routine dgesv: system is exactly singular: U[3,3] = 0
```

Or:

```
> det(XXdash)
[1] 0
```

**Attribution***Source : Link , Question Author : TsTeaTime , Answer Author : Robert Long*