Is it possible to have 1e-15 p-value when difference is about 1 SD?

The question is about the 2nd data table from the article Symptom improvement in children with autism spectrum disorder following bumetanide administration is associated with decreased GABA/glutamate ratios, specifically first row — total score.

Authors show the result of 37.27±4.09 (score in some test) in control group vs 34.51±3.35 in the group who took the medicine.

In the same row it’s stated that p-value for that is 3.46 × 10^{−15} even though the difference between groups is only about 1 SD.

So the question is: Are results specifically from the first row statistically significant? I have very basic understanding of statistics, but it seems to me that 1 SD is insignificant. Why do they have such a low p-value? Is it an “artifact” because they just multiplied probabilities?

Answer

The p 3.46 × 10−15 is not about the difference between groups but about the time × group interaction. The difference between groups has the p = 0.0012, which is in the text in the section Bumetanide improves ASD symptoms, and this p-value is correct given the numbers they provided. You can check it by plunging them into some t-test calculator.

The time × group interaction p-value is calculated using a random effect model and a permutation test, so it is not possible to just plug the numbers from the paper into a calculator and see if they match. Given the supplementary figure 1, which shows the data this test is testing, I wouldn’t be surprised that the p-value is extremely small, you can see that the symptom score for pretty much every control stays the same and the symptom score for every treatment case go down by a constant amount.

Although I think that the p-value is possible, I do not think that clinical data can show such a very regular pattern as shown in the supplementary figure, however, I know nothing about the field and barely even skimmed the paper.

edit
the figure in question:enter image description here

Attribution
Source : Link , Question Author : kirilloid , Answer Author : rep_ho

Leave a Comment