High Impact = High Statistical Standards? Not Necessarily So

Tressoldi, Patrizio E.; Giofré, David; Sella, Francesco; Cumming, Geoff
February 2013
PLoS ONE;Feb2013, Vol. 8 Issue 2, p1
Academic Journal
What are the statistical practices of articles published in journals with a high impact factor? Are there differences compared with articles published in journals with a somewhat lower impact factor that have adopted editorial policies to reduce the impact of limitations of Null Hypothesis Significance Testing? To investigate these questions, the current study analyzed all articles related to psychological, neuropsychological and medical issues, published in 2011 in four journals with high impact factors: Science, Nature, The New England Journal of Medicine and The Lancet, and three journals with relatively lower impact factors: Neuropsychology, Journal of Experimental Psychology-Applied and the American Journal of Public Health. Results show that Null Hypothesis Significance Testing without any use of confidence intervals, effect size, prospective power and model estimation, is the prevalent statistical practice used in articles published in Nature, 89%, followed by articles published in Science, 42%. By contrast, in all other journals, both with high and lower impact factors, most articles report confidence intervals and/or effect size measures. We interpreted these differences as consequences of the editorial policies adopted by the journal editors, which are probably the most effective means to improve the statistical practices in journals with high or low impact factors.


Related Articles

  • Meta-analysis is to analysis as meta-physics is to physics. Speroff, Leon // Contemporary OB/GYN;Jun2007, Vol. 52 Issue 6, p116 

    The author offers insights into the application of meta-analysis on individual clinical trials. He cites several limitations of the statistical method such that the meta-analysis of observational studies requires the authors to make subjective judgments as to which epidemiologic reports can be...

  • A Meta-Analysis of the Literature on Distal Radius Fractures. Paksima, Nader; Panchal, Anand; Posner, Martin A.; Green, Steven M.; Mehlman, Charles T.; Hiebert, Rudi // Bulletin of the NYU Hospital for Joint Diseases;2004, Vol. 62 Issue 1/2, p40 

    A structured meta-analysis of the available literature was performed to evaluate the outcome of the treatment of displaced intra-articular fractures of the distal radius. A comprehensive search of Medline using the key words "radius" and "fracture" revealed over 4,000 articles. After limiting...

  • Meta-Analysis Methods for Synthesizing Treatment Effects in Multisite Studies: Hierarchical Linear Modeling (HLM) Perspective. Kalaian, Sema A. // Practical Assessment, Research & Evaluation;2002-2003, Vol. 8, p1 

    A multisite study can be defined as the conduct of a research to study the effectiveness of similar or variations of the same interventions across multiple similar or distinct sites. These sites can involve multiple clusters of individuals, clinics, schools, classrooms, geographic locations,...

  • Numbers needed to treat derived from meta-analysis: a word of caution.  // ACP Journal Club;Jul/Aug2003, Vol. 139 Issue 1, pA-15 

    Comments on an editorial by Doctor Arthur Marx and Doctor Heiner Bucher in the March/April 2003 issue of 'ACP Journal Club' about meta-analysis. Baseline risk of included trials.

  • Meta-analysis of individual patient data from randomized trials: a review of methods used in practice. Simmonds, Mark C; Higgins, Julian PT; Stewart, Lesley A; Tierney, Jayne F; Clarke, Mike J; Thompson, Simon G // Clinical Trials;2005, Vol. 2 Issue 3, p209 

    Background Meta-analyses based on individual patient data (IPD) are regarded as the gold standard for systematic reviews. However, the methods used for analysing and presenting results from IPD meta-analyses have received little discussion. Methods We review 44 IPD meta-analyses published during...

  • Ataxia Rating Scales-Psychometric Profiles, Natural History and Their Application in Clinical Trials. Saute, Jonas; Donis, Karina; Serrano-Munuera, Carmen; Genis, David; Ramirez, Luís; Mazzetti, Pilar; Pérez, Luis; Latorre, Pilar; Sequeiros, Jorge; Matilla-Dueñas, Antoni; Jardim, Laura // Cerebellum;Jun2012, Vol. 11 Issue 2, p488 

    We aimed to perform a comprehensive systematic review of the existing ataxia scales. We described the disorders for which the instruments have been validated and used, the time spent in its application, its validated psychometric properties, and their use in studies of natural history and...

  • Meta-Analysis of Multi-Arm Trials Using Empirical Logistic Transform. Chootrakool, Hathaikan; Jian Qing Shi // Open Medical Informatics Journal;2008, Vol. 2, p112 

    Meta-analysis of multi-arm trials has been used increasingly in recent years. The aim of meta-analysis for multi-arm trials is to combine evidence from all possible similar studies. In this paper we propose normal approximation models by using empirical logistic transform to compare different...

  • Response to: Practical methods for incorporating summary time-to-event data into meta-analysis. Yu Wang; Tingting Zeng // Trials;2013, Vol. 14 Issue 1, p1 

    A response to and comment on Practical methods for incorporating summary time-to-event data into meta-analysis, by Jayne F Tierney et al.

  • Mechanisms of improved survival from intensive followup in colorectal cancer: a hypothesis. Renehan, A G; Egger, M; Saunders, M P; O'Dwyer, S T // British Journal of Cancer;2/14/2005, Vol. 92 Issue 3, p430 

    A meta-analysis of six randomised trials demonstrated that intensive followup in colorectal cancer was associated with an absolute reduction in all-cause 5-year mortality of 10%(95%confidence interval (CI): 4-16)-however, only two percent (95%CI: 0-5) was attributable to cure from salvage...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics