Evidence of bias and variation in diagnostic accuracy studies

Rutjes, Anne W. S.; Reitsma, Johannes B.; Di Nisio, Marcello; Smidt, Nynke; Van Rijn, Jeroen C.; Bossuyt, Patrick M. M.
February 2006
CMAJ: Canadian Medical Association Journal;2/14/2006, Vol. 174 Issue 4, special section p1
Academic Journal
Background: Studies with methodologic shortcomings can overestimate the accuracy of a medical test. We sought to determine and compare the direction and magnitude of the effects of a number of potential sources of bias and variation in studies on estimates of diagnostic accuracy. Methods: We identified meta-analyses of the diagnostic accuracy of tests through an electronic search of the databases MEDLINE, EMBASE, DARE and MEDION (1999-2002). We included meta-analyses with at least 10 primary studies without preselection based on design features. Pairs of reviewers independently extracted study characteristics and original data from the primary studies. We used a multivariable metaepidemiologic regression model to investigate the direction and strength of the association between 15 study features on estimates of diagnostic accuracy. Results: We selected 31 meta-analyses with 487 primary studies of test evaluations. Only 1 study had no design deficiencies. The quality of reporting was poor in most of the studies. We found significantly higher estimates of diagnostic accuracy in studies with nonconsecutive inclusion of patients (relative diagnostic odds ratio [RDOR] 1.5, 95% confidence interval [Cl] 1.0-2.1) and retrospective data collection (RDOR 1.6, 95% Cl 1.1-2.2). The estimates were highest in studies that had severe cases and healthy controls (RDOR 4.9, 95% Cl 0.6-37.3). Studies that selected patients based on whether they had been referred for the index test, rather than on clinical symptoms, produced significantly lower estimates of diagnostic accuracy (RDOR 0.5, 95% Cl 0.3-0.9). The variance between metaanalyses of the effect of design features was large to moderate for type of design (cohort v. case-control), the use of composite reference standards and the use of differential verification; the variance was close to zero for the other design features. Interpretation: Shortcomings in study design can affect estimates of diagnostic accuracy, but the magnitude of the effect may vary from one situation to another. Design features and clinical characteristics of patient groups should be carefully considered by researchers when designing new studies and by readers when appraising the results of such studies. Unfortunately, incomplete reporting hampers the evaluation of potential sources of bias in diagnostic accuracy studies.


Related Articles

  • Bridging the gaps in evidence based diagnosis. Straus, Sharon E. // BMJ: British Medical Journal (International Edition);8/26/2006, Vol. 333 Issue 7565, p405 

    The article discusses the difficulty of finding accurate evidence on the usefulness of diagnostic tests. Evidence based diagnostic sources are hard to find and the quality of evidence is low. The author proposes uses the "4S" pyramid created by Brian Haynes. The pyramid rates evidence by...

  • Combined Multichannel Intraluminal Impedance and Manometry Clarifies Esophageal Function Abnormalities: Study in 350 Patients. Tutuian, Radu; Castell, Donald O. // American Journal of Gastroenterology;Jun2004, Vol. 99 Issue 6, p1011 

    BACKGROUND: Combined multichannel intraluminal impedance and esophageal manometry (MII-EM) is a technique that uses an FDA-approved device allowing simultaneous evaluation of bolus transit (MII) in relation to pressure changes (EM).METHODS: During a 9-month period, beginning from July 2002...

  • Sources of bias in diagnostic accuracy studies and the diagnostic process. Furukawa, Toshi A.; Guyatt, Gordon H. // CMAJ: Canadian Medical Association Journal;2/14/2006, Vol. 174 Issue 4, p481 

    The section comments on the article "Evidence of Bias and Variation in Diagnostic Accuracy Studies," by A. W. Rutjes, et al, which appeared in the 2006 issue of the "Canadian Medical Association Journal." The limitations and strong points of the article are discussed. It emphasizes the...

  • TUBERCULOSIS AT AN UNUSUAL SITE. Haq, Zahidul; Mahbub, Md. Shahriar; Miah, Md. Titu; Alam, Md. Billal; Chowdhury, Riaz Ahmed; Ahasan, H. A. M. Nazmul // Journal of Medicine;Jan2010, Vol. 11 Issue 1, p67 

    Tuberculosis involving the anorectal region is a rare extrapulmonary form of the disease. We report two cases of isolated rectal tuberculosis with different presentations. On both instances, diagnosis was confirmed by histological examination and treated with anti-tuberculous drugs with...

  • A boy with acute urinary retention (2006: 3a). Verma, S. // European Radiology;Mar2006, Vol. 16 Issue 3, p767 

    The article calls for the diagnosis on the case of 10-year-old child with acute urinary retention. It presents the results of several clinical examinations performed to the patient including physical examination, routine laboratory tests and rectal examination. It invites readers to supply one...

  • Prenatal diagnosis of a de novo interstitial deletion of 11q (11q22.3 → q23.3) associated with abnormal ultrasound findings by array comparative genomic hybridization. Nian Liu; Jiong Yan; Xinlin Chen; Jieping Song; Bo Wang; Yanyi Yao // Molecular Cytogenetics (17558166);2014, Vol. 7 Issue 1, p1 

    Background Conventional G-band karyotyping offers low-resolution detection of chromosome abnormalities and cannot provide information about the involved genomic content. On the other hand, array comparative genomic hybridization can offer a rapid and comprehensive detection of genomewide gains...

  • 2 common exams that should never hurt. Hutcherson, Hilda // Glamour;Nov2004, Vol. 102 Issue 11, p113 

    Presents several medical examinations that never hurt the patient. Ways to have the correct pelvic examination; Methods for a painless rectal examination; Importance of communicating with the doctor.

  • Computer-aided detection for CT colonography: update 2007. Bielen, Didier; Kiss, Gabriel // Abdominal Imaging;Sep/Oct2007, Vol. 32 Issue 5, p571 

    Computed tomographic colonography (CTC) is an emerging technique for polyp detection in the colon. However, lesion detection can be challenging due to insufficient patient preparation, chosen CT technique or reader imperfection. The primary goal of computer-aided detection (CAD) for CTC is...

  • Variability in interpretation of chest radiographs among Russian clinicians and implications for screening programmes: observational study. Balabanova, Y.; Coker, R.; Fedorin, I.; Zakharova, S.; Plavinskij, S.; Krukov, N.; Atun, R.; Drobniewski, F. // BMJ: British Medical Journal (International Edition);8/13/2005, Vol. 331 Issue 7513, p379 

    Objective To determine variability in interpretation of chest radiographs among tuberculosis specialists, radiologists, and respiratory specialists. Design Observational study. Setting Tuberculosis and respiratory disease services, Samara region, Russian Federation. Participants 101 clinicians...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics