## Where does problems analytically arise in logistic regression on singular data matrix?

If you do linear regression without regularization on close-to-singular data matrix X (or it does not have enough data), the problem arises even in closed-form solution w=(XTX)−1XTy when you do (XTX)−1. If you do logistic regression, your gradient would look like: ∂L(w)∂w=X(y−logit(wTX)) and I don’t see any particular reason why it should diverge if X … Read more