Outcome selection bias in meta-analysis

Williamson, P. R.; Gamble, C.; Altman, D. G.; Hutton, J. L.
October 2005
Statistical Methods in Medical Research;Oct2005, Vol. 14 Issue 5, p515
Academic Journal
journal article
Publication bias has been previously identified as a threat to the validity of a meta-analysis. Recently, new evidence has documented an additional threat to validity, the selective reporting of trial outcomes within published studies. Several diseases have several possible measures of outcome. Some articles might report only a selection of those outcomes, perhaps those with statistically significant results. In this article, we review this problem while addressing the questions: what is within-study selective reporting? how common is it? why is it done? how can it mislead? how can it be detected?, and finally, what is the solution? We recommend that both publication bias and selective reporting should be routinely investigated in systematic reviews.


Related Articles

  • Appropriate Critical Percentages for the Schmidt and Hunter Meta-Analysis Procedure: Comparative Evaluation of Type I Error Rate and Power. Rasmussen, Jefrey Lee; Loher, Brian T. // Journal of Applied Psychology;Nov88, Vol. 73 Issue 4, p683 

    Recent investigations have shown that the Schmidt and Hunter 75% meta-analysis procedure (S&H-75%) does not have adequate control of the Type I error rate. This lack of control has caused two problems: First, the S&H-75% displays an erratic relationship between the likelihood to detect...

  • CONFIDENTIALITY ASSURANCES AND RESPONSE: A QUANTITATIVE REVIEW OF THE EXPERIMENTAL LITERATURE. Singer, Eleanor; von Thurn, Dawn R. // Public Opinion Quarterly;Spring95, Vol. 59 Issue 1, p66 

    This article tests the general hypothesis that a stronger assurance of confidentiality improves survey response by means of a meta-analysis of the experimental literature. No support is found for the general hypothesis, but the subsidiary hypothesis, that confidentiality assurances improve...

  • LESSONS LEARNED FROM THE TRENCHES OF METATRIANGULATION RESEARCH. Saunders, Carol S.; Jasperson, 'Jon; Butler, Brian S.; Carte, Traci A. // Communications of the Association for Information Systems;2003, Vol. 11, p245 

    In this paper we describe our application of a relatively untried research approach called metatriangulation. Metatriangulation is a three-phased, qualitative meta-analysis process that may be used to explore variations in the assumptions of alternative paradigms, gain insights into these...

  • Systematic review or meta-analysis? Cates, Chris // Update;6/16/2005, Vol. 70 Issue 6, p85 

    This article presents information on systematic reviews and meta-analyses. A systematic review simply gives a clear description of the methods used to search for studies and then to select which studies should be included, assess their quality and combine the results. This should be transparent...

  • Reviewing and Benchmarking Adventure Therapy Outcomes: Applications of Meta-Analysis. Neill, James T. // Journal of Experiential Education;2003, Vol. 25 Issue 3, p316 

    Summarizes evidence from meta-analyses of outdoor education, psychotherapy, and education which can be used to help determine the relative efficacy of adventure therapy programs. Key steps for advancing adventure therapy; Definition of effect sizes; Contextualization of adventure therapy...

  • Are Outcome Fairness and Outcome Favorability Distinguishable Psychological Constructs? A Meta-Analytic Review. Skitka, Linda J.; Winquist, Jennifer; Hutchinson, Susan // Social Justice Research;Dec2003, Vol. 16 Issue 4, p309 

    Manipulations of outcome favorability and outcome fairness are frequently treated as interchangeable, and assumed to have redundant effects. Perceptions of outcome fairness and outcome favorability are similarly presumed to have common antecedents and consequences. This research tested the...

  • A New Meta-Analytic Approach. Raju, Nambury S.; Normand, Jacques; Burke, Michael J.; Langlois, George M. // Journal of Applied Psychology;Jun91, Vol. 76 Issue 3, p432 

    A new meta-analytic approach for assessing the generalizability of correlations or criterion-related validities is presented. The new approach incorporates specific procedures for estimating the sampling variance of corrected and uncorrected correlations. A major advantage of the new am proach...

  • Metaanalyses—highest level of empirical evidence? Maier, Wolfgang; Möller, Hans-Jürgen // European Archives of Psychiatry & Clinical Neuroscience;2005, Vol. 255 Issue 6, p369 

    The article argues that metaanalyses cannot provide the highest level of evidence in evidence-based medicine. It explains how different methodological conditions will produce different results. It emphasizes that the results of metaanalyses depend on the method of how to identify the included...

  • Quality of Study Rating Form: An Instrument for Synthesizing Evaluation Studies. Gibbs, Leonard E. // Journal of Social Work Education;Winter1989, Vol. 25 Issue 1, p55 

    The Quality of Study Rating Form (QSRF) incorporates principles of meta-analysis to rate key features in an evaluation study. An Explanation for Criteria outlines the QSRF's sections regarding study quality, treatment effect size, and identification of treatment and client type. To measure...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics