statistical significance

Any comparison between two groups shows some difference, even when the thing being tested does nothing at all, because the people, places or samples in each group are never identical. A significance test asks how likely a difference at least as large as the one observed would be if the intervention had no effect. That starting assumption of no effect is called the null hypothesis.

The answer comes back as a p value, which the National Institute of Standards and Technology defines as the probability of a test statistic being at least as extreme as the one observed given that the null hypothesis is true. A small p value is an indication that the null hypothesis is false. Researchers fix a threshold in advance, the significance level, and call a result statistically significant when the p value falls below it. A level of 0.05 is the common convention, and it implies that a true null hypothesis is rejected 5 percent of the time; 0.1 and 0.01 are also used, and NIST notes the choice is somewhat arbitrary.

Two cautions matter for a reader. Statistically significant does not mean large or important: with enough data a trivial difference clears the threshold, and with very little data a real and useful effect can miss it. And a result reported as not statistically significant is not evidence that nothing happened, only that the study could not separate what it saw from chance. That is why a figure is best read alongside its confidence interval, which says how large the effect plausibly might be, rather than on the significance verdict alone.