What is the definition of a symmetric distribution?

What’s the definition of a symmetric distribution? Someone told me that a random variable $X$ came from a symmetric distribution if and only if $X$ and $-X$ has the same distribution. But I think this definition is partly true. Because I can present a counterexample $X\sim N(\mu,\sigma^{2})$ and $\mu\neq0$.
Obviously, it has a symmetric distribution, but $X$ and $-X$ have different distribution!
Am I right? Do you guys ever think about this question? What’s the exact definition of symmetric distribution?

Briefly: $X$ is symmetric when $X$ and $2a-X$ have the same distribution for some real number $a$. But arriving at this in a fully justified manner requires some digression and generalizations, because it raises many implicit questions: why this definition of “symmetric”? Can there be other kinds of symmetries? What is the relationship between a distribution and its symmetries, and conversely, what is the relationship between a “symmetry” and those distributions that might have that symmetry?

The symmetries in question are reflections of the real line. All are of the form

for some constant $a$.

So, suppose $X$ has this symmetry for at least one $a$. Then the symmetry implies

showing that $a$ is a median of $X$. Similarly, if $X$ has an expectation, then it immediately follows that $a = E[X]$. Thus we usually can pin down $a$ easily. Even if not, $a$ (and therefore the symmetry itself) is still uniquely determined (if it exists at all).

To see this, let $b$ be any center of symmetry. Then applying both symmetries we see that $X$ is invariant under the translation $x \to x + 2(b-a)$. If $b-a \ne 0$, the distribution of $X$ must have a period of $b-a$, which is impossible because the total probability of a periodic distribution is either $0$ or infinite. Thus $b-a=0$, showing that $a$ is unique.

More generally, when $G$ is a group acting faithfully on the real line (and by extension on all its Borel subsets), we could say that a distribution $X$ is “symmetric” (with respect to $G$) when

for all measurable sets $E$ and elements $g \in G$, where $E^g$ denotes the image of $E$ under the action of $g$.

As an example, let $G$ still be a group of order $2$, but now let its action be to take the reciprocal of a real number (and let it fix $0$). The standard lognormal distribution is symmetric with respect to this group. This example can be understood as an instance of a reflection symmetry where a nonlinear re-expression of the coordinates has taken place. This suggests focusing on transformations that respect the “structure” of the real line. The structure essential to probability must be related to Borel sets and Lebesgue measure, both of which can be defined in terms of (Euclidean) distance between two points.

A distance-preserving map is, by definition, an isometry. It is well known (and easy, albeit a little involved, to demonstrate) that all isometries of the real line are generated by reflections. Whence, when it is understand that “symmetric” means symmetric with respect to some group of isometries, the group must be generated by at most one reflection and we have seen that reflection is uniquely determined by any symmetric distribution with respect to it. In this sense, the preceding analysis is exhaustive and justifies the usual terminology of “symmetric” distributions.

Incidentally, a host of multivariate examples of distributions invariant under groups of isometries is afforded by considering “spherical” distributions. These are invariant under all rotations (relative to some fixed center). These generalize the one-dimensional case: the “rotations” of the real line are just the reflections.

Finally, it is worth pointing out that a standard construction–averaging over the group–gives a way to produce loads of symmetric distributions. In the case of the real line, let $G$ be generated by the reflection about a point $a$, so that it consists of the identity element $e$ and this reflection, $g$. Let $X$ be any distribution. Define the distribution $Y$ by setting

for all Borel sets $E$. This is manifestly symmetric and it’s easy to check that it remains a distribution (all probabilities remain nonnegative and the total probability is $1$).

Illustrating the group averaging process, the PDF of a symmetrized Gamma distribution (centered at $a=2$) is shown in gold. The original Gamma is in blue and its reflection is in red.