As far as I understand, principal components are obtained by rotating the coordinate axes to align them with the directions of maximum variance.
Nevertheless, I keep reading about “unrotated principal components” and my statistics software (SAS) gives me varimax-rotated principal components as well as the unrotated ones. Here I am confused: when we compute principal components, the axes are already rotated; so why is there another rotation needed? And what does “unrotated principal component” mean?
This is going to be a non-technical answer.
You are right: PCA is essentially a rotation of the coordinate axes, chosen such that each successful axis captures as much variance as possible.
In some disciplines (such as e.g. psychology), people like to apply PCA in order to interpret the resulting axes. I.e. they want to be able to say that principal axis #1 (which is a certain linear combination of original variables) has some particular meaning. To guess this meaning they would look at the weights in the linear combination. However, these weights are often messy and no clear meaning can be discerned.
In these cases, people sometimes choose to tinker a bit with the vanilla PCA solution. They take certain number of principal axes (that are deemed “significant” by some criterion), and additionally rotate them, trying to achieve some “simple structure” — that is, linear combinations that would be easier to interpret. There are specific algorithms that look for the simplest possible structure; one of them is called varimax. After varimax rotation, successive components do not anymore capture as much variance as possible! This feature of PCA gets broken by doing the additional varimax (or any other) rotation.
So before applying varimax rotation, you have “unrotated” principal components. And afterwards, you get “rotated” principal components. In other words, this terminology refers to the post-processing of the PCA results and not to the PCA rotation itself.
All of this is somewhat complicated by the fact that what gets rotated are loadings and not principal axes as such. However, for the mathematical details I refer you (and any interested reader) to my long answer here: Is PCA followed by a rotation (such as varimax) still PCA?