In this famous post “Gaussian Distributions are Soap Bubbles” it is claimed that the distribution of the points looks like a soap bubble (where it is less dense in the center and more dense at the edge) instead of a bold of mold where it is more dense in the center. I would expect that it is more dense in the center like it is in two or three dimensional.
From the post, I could not figure out why this is the case. It uses three figures which I could not figure out what they are telling.
Can someone clarify this figures and the main question why it is looking like a soap bubble in high dimensions?
I can’t answer about what the OP’s famous post claims, but let us consider the simpler case of uniform distributions on the unit disc: (X,Y) is uniformly distributed on the unit disc (that is, f(X,Y)(x,y)=1π for x2+y2<1. What is the probability that (X,Y) is closer to the unit circle, that is, closer to the boundary of the unit disc than it is to the origin (center of the circle)? Well, only those points that lie inside the circle of radius 12 are at distance <12 from the origin, and so all points outside this smaller circle are at distance >12 from the origin. It is an easy computation to arrive at
A similar calculation for a uniform distribution on the interior of a unit sphere in 3 dimensions (the pdf has value 34π on the interior) gives
Generalizing to n>3 dimensions and remembering that the volume of an n-dimeensional hypersphere or radius r is proportional to rn, we get by very similar calculations that
that is, most of the probability mass_ lies closer to the surface of the sphere than to the origin. As a final comment, note that the Xi are NIBNID random variables which acronym stands for Not Independent But Nonetheless Identically Distributed.
Turning to IID standard Gaussian random variables, the joint density is not uniformly distributed but has a very pronounced peak at the origin. But, there is so little volume near the center of a hypersphere as compared to closer to the surface that when we integrate the density over the volume of a hypersphere of small radius r to find P(√∑ni=1X2i<r), most of this probability mass is obtained from the small contributions from the periphery (there are so many of them) and very little from the few but larger contributions from the core; that is, most of the probability mass lies closer to the skin of the orange than to the center. But things change as r increases. Since ∑ni=1X2i is a χ2 random variable with n degrees of freedom (with mean n and variance 2n), which for large n can be approximated as a Gaussian random variable with the same mean and variance) most of its probability was lies in the range [n−√18n,n−√18n]=[μ−3σ,μ+3σ]. Put another way,
the quantity P(∑ni=1X2i<r2) is close to 0 for small r (the nearly empty space inside the soap bubble), and then (regarded as a function of r) increases very rapidly with r in the close vicinity of r=√n (this is the thin skin of the bubble where most of the mass is) to almost 1, and then very slowly to its asymptotic value of 1 (the nearly empty space outside the bubble). In short, the soap bubble analogy is very apt for Gaussian distributions; almost all the probability mass of the joint pdf of n standard Gaussian random variables does indeed lie in a very thin shell of radius ≈√n and there is very little probability mass that is not in the shell -- both the interior and the exterior of the shell is mostly empty as is the case with soap bubbles.