Using Standard Studies to Interpret the Scores from Scales for Assessing the Internal Validity of Randomized Controlled Trials

Slack, Marion Kimball
November 2007
Hospital Pharmacy;Nov2007, Vol. 42 Issue 11, p1027
Academic Journal
Objective: A large number of tools for assessing the quality of randomized controlled trials are available; however, users have little guidance as to whether a given score represents high or low validity. The purpose of this study is to explore the use of studies identified as having high-internal validity, referred to as the standard studies, to interpret internal validity scores from studies with unknown internal validity. Methods: The standard studies were identified by locating 6 candidate studies reporting the findings of randomized controlled trials from the Journal of American Medicine Association or the New England Journal of Medicine and scoring the studies using 2 scales, the Jadad scale (high score = 5; low = 0) and an internal validity information scale (IVI; high score = 70; low = 0). The 2 studies with the highest average rank were chosen as the standard studies. To determine if the standard studies facilitate interpretation of internal validity scores, 11 randomized controlled trials of garlic for the treatment of hypercholesterolemia were scored using both the Jadad and IVI scales and differences in scores resolved by consensus. The total mean scores were compared with the standard-study mean using a 1-sample t test; the interpretation was based on the comparison. Results: The mean-internal validity score for the 2 standard studies was 4.5(90%) for the Jadad scale and 64 (91%) for the IVI scale. For tile garlic studies, the Jadad scores ranged from 3 to 5 (60% to 80%), tile IVI scores from 15 to 59 (21% to 84%); the mean scores were 4 (standard deviation = 0.9; 80%) and 37(53%), respectively. On the IVI scale, tile garlic studies had lower internal validity (P < 0.001), a finding substantiated by the low scores for addressing withdrawals, blinding, and adherence. On the Jadad scale, the scores were comparable with the score for the standard studies (P = 0.093). Conclusions: The strategy of using the standard studies m interpret the scores for the IVI scale worked well; the comparison indicated that, overall, the garlic studies were of low-internal validity. With the Jadad scale, the strategy resulted in erroneous conclusions that the garlic studies had relatively high-internal validity; however, it did indicate that the Jadad scale is limited and may not identify studies with very low-internal validity. The results of this study suggest that scales used to assess internal validity should be tested on studies with a range of validity, so the functional characteristics of the scale can be identified.


Related Articles

  • Guest Editorial Sample Size Considerations for Restoration-Longevity Randomized Controlled Trials. van Landuyt, Kirsten; Fieuws, Steffen; van Meerbeek, Bart // Journal of Adhesive Dentistry;2008, Vol. 10 Issue 4, p247 

    The author reflects on issues related to clinical randomized controlled trial (RCT) as a research method to determine the effectiveness of dental restorative materials. He cites the arguments of clinical researchers that such method is time consuming and that often contradicting aspects need to...

  • Clinical trials in India: Where do we stand globally? Selvarajan, Sandhiya; George, Melvin; S., Suresh Kumar; Dkhar, Steven Aibor // Perspectives in Clinical Research;Jul-Sep2013, Vol. 4 Issue 3, p160 

    Aims: To evaluate the trend of clinical trials in India over the last 4 years compared to the well-established countries using clinical trial registries since the advent of clinical trial registry of India (CTRI). Materials and Methods: The data of clinical trials registered in India, United...

  • Conflicts of Interest at Medical Journals: The Influence of Industry-Supported Randomised Trials on Journal Impact Factors and Revenue - Cohort Study. Lundh, Andreas; Barbateskovic, Marija; Hróbjartsson, Asbjørn; Gøtzsche, Peter C. // PLoS Clinical Trials;Oct2010, Vol. 7 Issue 10, Special section p1 

    Background: Transparency in reporting of conflict of interest is an increasingly important aspect of publication in medical journals. Publication of large industry-supported trials may generate many citations and journal income through reprint sales and thereby be a source of conflicts of...

  • Exploring the black box of change in improving test-ordering routines. Marloes A van Bokhoven; Hèlen Koch; Geert-Jan Dinant; Patrick JE Bindels; Richard PTM Grol; Trudy van der Weijden // Family Practice;Jun2008, Vol. 25 Issue 3, p139 

    Background. The effects of quality improvement strategies are sometimes limited in spite of a systematic development approach. What elements play a role in the change process is not yet fully understood. Objective. To explore this ‘black box’ of change, by analysing the barriers and...

  • Results of a phase III clinical trial: CHOP versus CMED in peripheral T-cell lymphoma unspecified. Claudia Castañeda; Natividad Neri; Sergio Cleto; Alejandra Talavera; Martha González; Judith Huerta-Guzmán; M. Nambo // Medical Oncology;Sep2008, Vol. 25 Issue 3, p360 

    Abstract  We performed a controlled clinical trial to define the use of a brief therapy: CMED (cyclophosphamide, etoposide, methotrexate, and dexamethasone) compared with standard CHOP (cyclophosphamide, doxorubicin, vincristine, and prednisone) in the treatment of peripheral T-cell...

  • Continuing aripiprazole after stabilisation of a manic or mixed episode of bipolar I disorder delays relapse. Keck, P. E.; Calabrese, J. R.; McQuade, R. D. // Evidence Based Mental Health;Feb2007, Vol. 10 Issue 1, p13 

    The article discusses the findings of a medical research showing the effectiveness of aripiprazole with bipolar disorder after been stabilized with aripiprazole in Great Britain. It was stated that aripiprazole holds the revision in people recently been established or mixed with the element...

  • Comparative Effectiveness Research and CAM. Aickin, Mikel // Journal of Alternative & Complementary Medicine;Jan2010, Vol. 16 Issue 1, p1 

    The author reflects on the efficacy of the comparative effectiveness research (CAM) in the U.S. The author mentions that the scheme is highly criticized in the industry because CAM is generally done on biomedical frontier where conventional biomedicine works poorly. Moreover, he believes that...

  • Obstacles to researching the researchers: A case study of the ethical challenges of undertaking methodological research investigating the reporting of randomised controlled trials. McKenzie, Joanne E.; Herbison, G. Peter; Roth, Paul; Paul, Charlotte // Trials;2010, Vol. 11, p28 

    Background: Recent cohort studies of randomised controlled trials have provided evidence of within-study selective reporting bias; where statistically significant outcomes are more likely to be more completely reported compared to non-significant outcomes. Bias resulting from selective reporting...

  • Commentary: External validity of results of randomized trials: disentangling a complex concept. Rothwell, Peter M. // International Journal of Epidemiology;Feb2010, Vol. 39 Issue 1, p94 

    The author discusses his thesis regarding the external validity of randomized controlled trial results (RCT) which he refers to as the ability of a result to be replicated in a definable group of patients in a hospital setting. He explains the complexity behind proving the external validity of...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics