Paper detail

The evidence contained in the P-value is context dependent

In a recent opinion article, Muff et al. recapitulate well-known objections to the Neyman-Pearson Null-Hypothesis Significance Testing (NHST) framework and call for reforming our practices in statistical reporting. We agree with them on several important points: the significance threshold P<0.05 is only a convention, chosen as a compromise between type I and II error rates; transforming the p-value into a dichotomous statement leads to a loss of information; and p-values should be interpreted together with other statistical indicators, in particular effect sizes and their uncertainty. In our view, a lot of progress in reporting results can already be achieved by keeping these three points in mind. We were surprised and worried, however, by Muff et al.&#39;s suggestion to interpret the p-value as a &#34;gradual notion of evidence&#34;. Muff et al. recommend, for example, that a P-value > 0.1 should be reported as &#34;little or no evidence&#34; and a P-value of 0.001 as &#34;strong evidence&#34; in favor of the alternative hypothesis H1.