TITLE

The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey

AUTHOR(S)
Ioannidis, John P. A.; Trikalinos, Thomas A.
PUB. DATE
April 2007
SOURCE
CMAJ: Canadian Medical Association Journal;4/10/2007, Vol. 176 Issue 8, p1091
SOURCE TYPE
Academic Journal
DOC. TYPE
Article
ABSTRACT
Background: Statistical tests for funnel-plot asymmetry are common in meta-analyses. Inappropriate application can generate misleading inferences about publication bias. We aimed to measure, in a survey of meta-analyses, how frequently the application of these tests would be not meaningful or inappropriate. Methods: We evaluated all meta-analyses of binary outcomes with ≥ 3 studies in the Cochrane Database of Systematic Reviews (2003, issue 2). A separate, restricted analysis was confined to the largest meta-analysis in each of the review articles. In each meta-analysis, we assessed whether criteria to apply asymmetry tests were met: no significant heterogeneity, I2 < 50%, ≥ 10 studies (with statistically significant results in at least 1) and ratio of the maximal to minimal variance across studies > 4. We performed a correlation and 2 regression asymmetry tests and evaluated their concordance. Finally, we sampled 60 meta-analyses from print journals in 2005 that cited use of the standard regression test. Results: A total of 366 of 6873 (5%) and 98 of 846 metaanalyses (12%) in the wider and restricted Cochrane data set, respectively, would have qualified for use of asymmetry tests. Asymmetry test results were significant in 7%-18% of the meta-analyses. Concordance between the 3 tests was modest (estimated κ 0.33-0.66). Of the 60 journal metaanalyses, 7 (12%) would qualify for asymmetry tests; all 11 claims for identification of publication bias were made in the face of large and significant heterogeneity. Interpretation: Statistical conditions for employing asymmetry tests for publication bias are absent from most metaanalyses; yet, in medical journals these tests are performed often and interpreted erroneously.
ACCESSION #
24577256

 

Related Articles

  • Discussion. Chambers, Ray // Journal of Official Statistics (JOS);Sep2015, Vol. 31 Issue 3, p507 

    The author comments on the articles by J. Burger et al, S. Gerritse et al and T. Tuto and L. Di Consiglio which tackle measurement-error issues that are important emerging features of official statistics methodology.Topics discusses include their common focus on problems that arise when...

  • The offering of family presence during resuscitation: a systematic review and meta-analysis. Oczkowski, Simon J. W.; Mazzetti, Ian; Cupido, Cynthia; Fox-Robichaud, Alison E. // Journal of Intensive Care;10/14/2015, Vol. 3, p1 

    Background: Family members may wish to be present during resuscitation of loved ones, despite concerns that they may interfere with the resuscitation or experience psychological harm. Methods: We conducted a systematic review to determine whether offering family presence during resuscitation...

  • Fitting and Interpreting Occupancy Models. Welsh, Alan H.; Lindenmayer, David B.; Donnelly, Christine F. // PLoS ONE;Jan2013, Vol. 8 Issue 1, Special section p1 

    We show that occupancy models are more difficult to fit than is generally appreciated because the estimating equations often have multiple solutions, including boundary estimates which produce fitted probabilities of zero or one. The estimates are unstable when the data are sparse, making them...

  • CURRENT KNOWLEDGE AND CONSIDERATIONS REGARDING SURVEY REFUSALS. DUTWIN, DAVID; LOFT, JOHN D.; DARLING, JILL E.; HOLBROOK, ALLYSON L.; JOHNSON, TIMOTHY P.; LANGLEY, RONALD E.; LAVRAKAS, PAUL J.; OLSON, KRISTEN; PEYTCHEVA, EMILIA; STEC, JEFFERY A.; TRIPLETT, TIMOTHY; ZUKERBERG, ANDREW // Public Opinion Quarterly;Summer2015, Vol. 79 Issue 2, p411 

    The article presents a report from the American Association for Public Opinion Research (AAPOR) Task Force on survey response and refusal. The author reflect on the trend of rising rates of refusal to participation in surveys and the effect of refusal bias on the validity of survey results....

  • A meta-analytic review of leadership impact research: Experimental and quasi-experimental studies Avolio, Bruce J.; Reichard, Rebecca J.; Hannah, Sean T.; Walumbwa, Fred O.; Chan, Adrian // Leadership Quarterly;Oct2009, Vol. 20 Issue 5, p764 

    In this study we set out to conduct a comprehensive quantitative research analysis of literature reporting results on the causal impact of leadership by focusing on examining what we refer to as ‘leadership interventions.’ We defined leadership interventions as those studies where...

  • Research on Disability in Thailand: Meta-Analysis and Qualitative Analysis. Cheausuwantavee, Tavee; Nookaew, Sripair; Cheausuwantavee, Chommanad // International Journal of Interdisciplinary Social Sciences;2010, Vol. 5 Issue 1, p311 

    This study aimed to synthesize the researches regarding disabilities in Thailand in which had been done during 1997-2009 and then those were analyzed by meta-analysis and content analysis. The target researches were collected both electronic database and hard documents that were available in the...

  • Maturity Models in Information Systems Research: Literature Search and Analysis. Pöppelbuß, Jens; Niehaves, Björn; Simons, Alexander; Becker, Jörg // Communications of the Association for Information Systems;2011, Vol. 29, p505 

    This article collects and analyzes seventy-six maturity model articles that have been published in leading Information Systems (IS) journals and conference proceedings during the past fifteen years. We study the IS literature on maturity models from three different perspectives: a research...

  • A general framework for comparative Bayesian meta-analysis of diagnostic studies. Menten, Joris; Lesaffre, Emmanuel // BMC Medical Research Methodology;Aug2015, Vol. 15 Issue 1, p1 

    Background: Selecting the most effective diagnostic method is essential for patient management and public health interventions. This requires evidence of the relative performance of alternative tests or diagnostic algorithms. Consequently, there is a need for diagnostic test accuracy...

  • And the winner is...an opinion poll that adds up. Kiernan, Vincent // New Scientist;04/26/97, Vol. 154 Issue 2079, p5 

    Presents a technique developed by Gary King of Harvard University in Cambridge, Massachusetts which utilizes ecological inference in predicting election results. Lack of accuracy of results of opinion polls; Basis of King's method.

Share

Read the Article

Courtesy of VIRGINIA BEACH PUBLIC LIBRARY AND SYSTEM

Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics