Can a statistical test return a p-value of zero?

I don’t mean a value close to zero (rounded to zero by some statistical software) but rather a value of literally zero. If so, would it mean that the probability of getting the obtained data assuming the null hypothesis is true is also zero? What are (some examples) of statistical tests that can return results of this sort?

Edited the second sentence to remove the phrase “the probability of the null hypothesis”.

Answer

It will be the case that if you observed a sample that’s impossible under the null (and if the statistic is able to detect that), you can get a p-value of exactly zero.

That can happen in real world problems. For example, if you do an Anderson-Darling test of goodness of fit of data to a standard uniform with some data outside that range – e.g. where your sample is (0.430, 0.712, 0.885, 1.08) – the p-value is actually zero (but a Kolmogorov-Smirnov test by contrast would give a p-value that isn’t zero, even though we can rule it out by inspection).

Likelihood ratio tests will likewise give a p-value of zero if the sample is not possible under the null.

As whuber mentioned in comments, hypothesis tests don’t evaluate the probability of the null hypothesis (or the alternative).

We don’t (can’t, really) talk about the probability of the null being true in that framework (we can do it explicitly in a Bayesian framework, though — but then we cast the decision problem somewhat differently from the outset).

Attribution
Source : Link , Question Author : user1205901 – Слава Україні , Answer Author : Glen_b

Leave a Comment