Why is controlling FDR less stringent than controlling FWER?

I have read that controlling FDR is less stringent than controlling FWER, such as in Wikipedia:

FDR controlling procedures exert a less stringent control over false discovery compared to familywise error rate (FWER) procedures (such as the Bonferroni correction). This increases power at the cost of increasing the rate of type I errors, i.e., rejecting the null hypothesis of no effect when it should be accepted.

But I was wondering how it is shown to be true mathematically?

Is there some relation between FDR and FWER?

Answer

Indeed, @cardinal is quite right that the paper is as clear as it gets. So, for what it’s worth, in case you do not have access to the paper, here’s a slightly elaborated version of how Benjamini–Hochberg argue:

The FDR Qe is the expected value of the proportion of false rejections v to all rejections r. Now, r is, obviously, the sum of false and correct rejections; call the latter s.

In summary, (using capital letters for random variables and lowercase letters for realized values),

Qe=E(VR)=E(VV+S)=:E(Q).

One takes Q=0 if R=0.

Now, there are two possibilities: either all m nulls are true or just m0<m of them are true. In the first case, there cannot be correct rejections, so r=v. Thus, if there are any rejections (r1), q=1, otherwise q=0. Hence,

\newcommand{\FDR}{\mathrm{FDR}}\newcommand{\FWER}{\mathrm{FWER}}\FDR=E(Q)=1\cdot P(Q=1)+0\cdot P(Q=0)=P(Q=1)=P(V \geq 1)=\FWER

So, \FDR=\FWER in this case, such that any procedure that controls the \FDR trivially also controls the \FWER and vice versa.

In the second case in which m_0<m, if v>0 (so if there is at least one false rejection), we obviously have (this being a fraction with also v in the denominator) that v/r\leq 1. This implies that the indicator function that takes the value 1 if there is at least one false rejection, \mathbf 1_{V\geq 1} will never be less than Q, \mathbf 1_{V\geq 1}\geq Q. Now, take expectation on either side of the inequality, which by monotonicity of E leaves the inequality intact,

E(\mathbf 1_{V\geq 1})\geq E(Q)=\FDR

The expected value of an indicator function being the probability of the event in the indicator, we have E(\mathbf 1_{V\geq 1})=P(V\geq 1), which again is the \FWER.

Thus, when we have a procedure that controls the \FWER in the sense that \FWER\leq \alpha, we must have that \FDR\leq\alpha.

Conversely, having \FDR control at some \alpha may come with a substantially larger \FWER. Intuitively, accepting a nonzero expected fraction of false rejections (\FDR) out of a potentially large total of hypotheses tested may imply a very high probability of at least one false rejection (\FWER).

So, a procedure has to be less strict when only \FDR control is desired, which is also good for power. This is the same idea as in any basic hypothesis test: when you test at the 5% level you reject more frequently (both correct and false nulls) than when testing at the 1% level simply because you have a smaller critical value.

Attribution
Source : Link , Question Author : Tim , Answer Author : cardinal

Leave a Comment