Do you reject the null hypothesis when p<αp < \alpha or p≤αp \leq \alpha?

This is clearly just a matter of definition or convention, and of almost no practical importance. If α is set to its traditional value of 0.05, is a p value of 0.0500000000000… considered to be statistically significant or not? Is the rule to define statistical significance usually considered to be p<α or pα??

Answer

Relying on Lehmann and Romano, Testing Statistical Hypotheses, . Defining S1 as the region of rejection and ΩH as the null hypothesis region, loosely speaking, we have the following statement, p. 57 in my copy:

Thus one selects a number α between 0 and 1, called the level
of significance
, and imposes the condition that:

... Pθ{XS1}α for all θΩH

Since it is possible that Pθ{XS1}=α, it follows that you'd reject for p-values α.

On a more intuitive level, imagine a test on a discrete parameter space, and a best (most powerful) rejection region with a probability of exactly 0.05 under the null hypothesis. Assume the next largest (in terms of probability) best rejection region had a probability of 0.001 under the null hypothesis. It would be kind of difficult to justify, again intuitively speaking, saying that the first region was not equivalent to an "at the 95% level of confidence..." decision but that you had to use the second region to reach the 95% level of confidence.

Attribution
Source : Link , Question Author : Harvey Motulsky , Answer Author : chl

Leave a Comment