中国科学院行为科学重点实验室知识库

不显著结果(如,p>0.05)在心理学研究中十分常见,且容易被误解为接受零假设的证据,并可能导致分组匹配研究的错误推断或者忽视被小样本的不显著结果掩盖的真实效应。但国内目前尚无实证研究对不显著结果的普遍性及其解读进行调查。本研究调查500篇中文心理学实证研究,统计其摘要中出现与不显著结果相关的阴性陈述的频率,判断并统计基于阴性陈述的推断准确性,并使用贝叶斯因子对不显著结果中包含t值的研究进行重新评估。结果表明, 36%的摘要提及不显著结果,共包含236个阴性陈述。其中, 41%的阴性陈述对不显著结果的解读出现偏差(如,解读为支持了零假设)。对包含t值的研究进行贝叶斯因子分析,结果显示仅有5.1%的不显著结果可以提供强证据支持零假设(BF_(01)>10)。与先前对国际心理学期刊的调查结果相比(32%的摘要包含阴性陈述;72%的阴性陈述对不显著结果的解读错误),中文心理学期刊中报告不显著结果的比例更高,且对不显著结果解读错误的比例更低。但国内研究者仍需进一步加强对不显著结果的认识,推广适于评估不显著结果的统计方法。

Nonsignificant results are common in psychological research and can be easily misinterpreted as evidence for accepting null hypothesis. This misinterpretation may lead to false statistical inferences in empirical research. However, how prevalent this misinterpretation exists in Chinese published psychological studies is unknown. To answer this question, we randomly selected 500 empirical research papers published between 2017 and 2018 in Acta Psychological Sinica, Journal of Psychological Science, Chinese Journal of Clinical Psychology, Psychological Development and Education, Psychological and Behavioral Studies, screened articles in which the abstracts contained any sentences that indicated nonsignificant results (we call these sentences “negative statements” hereafter). We then read those articles and extracted negativestatements-related statistics and their interpretations, and evaluated the correctness of each interpretation. Finally, we calculated Bayes factors based on the available t values in these nonsignificant results. The protocol was pre-registered at OSF (https://osf.io/czx6f). We found that (1) out of 500 empirical research, 36% of their abstracts (n = 180) contained negative statements; (2) in those 180 articles, we extracted 236 nonsignificant results and corresponding interpretations, and found that 41% of these interpretations was incorrect, (3) Bayes factor analysis revealed that only 5.1% (n = 2) of available nonsignificant t-values (n =39) can provide strong evidence in favor of null hypothesis (BF01 > 10). We compared the results with Aczel et al. (2018) and discussed the potential reasons that caused the misinterpretation. These data suggest that Chinese psychology researchers need to improve their understanding of nonsignificant results and statistical inference.