# What’s the formula for the Benjamini-Hochberg adjusted p-value?

I understand the procedure and what it controls. So what’s the formula for the adjusted p-value in the BH procedure for multiple comparisons?

Just now I realized the original BH didn’t produce adjusted p-values, only adjusted the (non) rejection condition: https://www.jstor.org/stable/2346101. Gordon Smyth introduced adjusted BH p-values in 2002 anyways, so the question still applies. It’s implemented in R as p.adjust with method BH.

## Answer

The famous seminal Benjamini & Hochberg (1995) paper described the procedure for accepting/rejecting hypotheses based on adjusting the alpha levels. This procedure has a straightforward equivalent reformulation in terms of adjusted $$pp$$-values, but it was not discussed in the original paper. According to Gordon Smyth, he introduced adjusted $$pp$$-values in 2002 when implementing p.adjust in R. Unfortunately, there is no corresponding citation, so it has always been unclear to me what one should cite if one uses BH-adjusted $$pp$$-values.

Turns out, the procedure is described in the Benjamini, Heller, Yekutieli (2009):

An alternative way of presenting the results of this procedure is by presenting the adjusted $$pp$$-values. The BH-adjusted $$pp$$-values are defined as $$pBH(i)=minp^\mathrm{BH}_{(i)} = \min\Big\{\min_{j\ge i}\big\{\frac{mp_{(j)}}{j}\big\},1\Big\}.$$

This formula looks more complicated than it really is. It says:

1. First, order all $$pp$$-values from small to large. Then multiply each $$pp$$-value by the total number of tests $$mm$$ and divide by its rank order.
2. Second, make sure that the resulting sequence is non-decreasing: if it ever starts decreasing, make the preceding $$pp$$-value equal to the subsequent (repeatedly, until the whole sequence becomes non-decreasing).
3. If any $$pp$$-value ends up larger than 1, make it equal to 1.

This is a straightforward reformulation of the original BH procedure from 1995. There might exist an earlier paper that explicitly introduced the concept of BH-adjusted $$pp$$-values, but I am not aware of any.

Update. @Zenit found that Yekutieli & Benjamini (1999) described the same thing already back in 1999:

Attribution
Source : Link , Question Author : Firebug , Answer Author : amoeba