# Is it wrong to refer to results as being “highly significant”?

Why do statisticians discourage us from referring to results as “highly significant” when the $p$-value is well below the conventional $\alpha$-level of $0.05$?

Is it really wrong to trust a result that has 99.9% chance of not being a Type I error ($p=0.001$) more than a result that only gives you that chance at 99% ($p=0.01$)?

I think there is not much wrong in saying that the results are “highly significant” (even though yes, it is a bit sloppy).

It means that if you had set a much smaller significance level $\alpha$, you would still have judged the results as significant. Or, equivalently, if some of your readers have a much smaller $\alpha$ in mind, then they can still judge your results as significant.

Note that the significance level $\alpha$ is in the eye of the beholder, whereas the $p$-value is (with some caveats) a property of the data.

Observing $p=10^{-10}$ is just not the same as observing $p=0.04$, even though both might be called “significant” by standard conventions of your field ($\alpha=0.05$). Tiny $p$-value means stronger evidence against the null (for those who like Fisher’s framework of hypothesis testing); it means that the confidence interval around the effect size will exclude the null value with a larger margin (for those who prefer CIs to $p$-values); it means that the posterior probability of the null will be smaller (for Bayesians with some prior); this is all equivalent and simply means that the findings are more convincing. See Are smaller p-values more convincing? for more discussion.

The term “highly significant” is not precise and does not need to be. It is a subjective expert judgment, similar to observing a surprisingly large effect size and calling it “huge” (or perhaps simply “very large”). There is nothing wrong with using qualitative, subjective descriptions of your data, even in the scientific writing; provided of course, that the objective quantitative analysis is presented as well.