Inspired by my other question, I would like to ask how does one find the mode of a probability density function (PDF) of a function f(x)?
Is there any “cook-book” procedure for this? Apparently, this task is much more difficult than it seems at first.
Saying “the mode” implies that the distribution has one and only one. In general a distribution may have many modes, or (arguably) none.
If there’s more than one mode you need to specify if you want all of them or just the global mode (if there is exactly one).
Assuming we restrict ourselves to unimodal distributions*, so we can speak of “the” mode, they’re found in the same way as finding maxima of functions more generally.
*note that page says “as the term “mode” has multiple meanings, so does the term “unimodal”” and offers several definitions of mode — which can change what, exactly, counts as a mode, whether there is 0 1 or more — and also alters the strategy for identifying them. Note particularly how general the “more general” phrasing of what unimodality is in the opening paragraph “unimodality means there is only a single highest value, somehow defined“
One definition offered on that page is:
A mode of a continuous probability distribution is a value at which the probability density function (pdf) attains its maximum value
So given a specific definition of the mode you find it as you would find that particular definition of “highest value” when dealing with functions more generally, (assuming that the distribution is unimodal under that definition).
There are a variety of strategies in mathematics for identifying such things, depending on circumstances. See, the “Finding functional maxima and minima” section of the Wikipedia page on Maxima and minima which gives a brief discussion.
For example, if things are sufficiently nice — say we’re dealing with a continuous random variable, where the density function has continuous first derivative — you might proceed by trying to find where the derivative of the density function is zero, and checking which type of critical point it is (maximum, minimum, horizontal point of inflexion). If there’s exactly one such point which is a local maximum, it should be the mode of a unimodal distribution.
However, in general things are more complicated (e.g. the mode may not be a critical point), and the broader strategies for finding maxima of functions come in.
Sometimes, finding where derivatives are zero algebraically may be difficult or at least cumbersome, but it may still be possible to identify maxima in other ways. For example, it may be that one might invoke symmetry considerations in identifying the mode of a unimodal distribution. Or one might invoke some form of numerical algorithm on a computer, to find a mode numerically.
Here are some cases that illustrate typical things that you need to check for – even when the function is unimodal and at least piecewise continuous.
So, for example, we must check endpoints (center diagram), points where the derivative changes sign (but may not be zero; first diagram), and points of discontinuity (third diagram).
In some cases, things may not be so neat as these three; you have to try to understand the characteristics of the particular function you’re dealing with.
I haven’t touched on the multivariate case, where even when functions are quite “nice”, just finding local maxima may be substantially more complex (e.g. the numerical methods for doing so can fail in a practical sense, even when they logically must succeed eventually).