Surgeons’ Non-technical Skills in the Operating Room: Reliability Testing of the NOTSS Behavior Rating System

Yule, Steven; Flin, Rhona; Maran, Nicola; Rowley, David; Youngson, George; Paterson-Brown, Simon
April 2008
World Journal of Surgery;Apr2008, Vol. 32 Issue 4, p548
Academic Journal
Previous research has shown that surgeons’ intraoperative non-technical skills are related to surgical outcomes. The aim of this study was to evaluate the reliability of the NOTSS (Non-technical Skills for Surgeons) behavior rating system. Based on task analysis, the system incorporates five categories of skills for safe surgical practice (Situation Awareness, Decision Making, Task Management, Communication & Teamwork, and Leadership). Consultant (attending) surgeons ( n = 44) from five Scottish hospitals attended one of six experimental sessions and were trained to use the NOTSS system. They then used the system to rate consultant surgeons’ behaviors in six simulated operating room scenarios that were presented using video. Surgeons’ ratings of the behaviors demonstrated in each scenario were compared to expert ratings (“accuracy”), and assessed for inter-rater reliability and internal consistency. The NOTSS system had a consistent internal structure. Although raters had minimal training, rating “accuracy” for acceptable/unacceptable behavior was above 60% for all categories, with mean of 0.67 scale points difference from reference (expert) ratings (on 4-point scale). For inter-rater reliability, the mean values of within-group agreement ( r wg) were acceptable for the categories Communication & Teamwork (.70), and Leadership (.72), but below a priori criteria for other categories. Intra-class correlation coefficients (ICC) indicated high agreement using average measures (values were .95–.99). With the requisite training, the prototype NOTSS system could be used reliably by surgeons to observe and rate surgeons’ behaviors. The instrument should now be tested for usability in the operating room.


Related Articles

  • Patient undergoing after-hours surgery at heightened risk of reoperations.  // Medical Device Daily;11/7/2009, p1 

    The article discusses research which revealed that patients who underwent after-hour orthopedic surgeries face a slightly higher rate of necessary follow-up surgeries. It references a study by William Ricci and colleagues published in the September 2009 issue of "The Journal of Bone and Joint...

  • Improvement of teamwork and safety climate following implementation of the WHO surgical safety checklist at a university hospital in Japan. Kawano, Takashi; Taniwaki, Miki; Ogata, Kimiyo; Sakamoto, Miwa; Yokoyama, Masataka // Journal of Anesthesia;Jun2014, Vol. 28 Issue 3, p467 

    With the aim to optimize surgical safety, the World Health Organization (WHO) introduced the Surgical Safety Checklist (SSCL) in 2008. The SSCL has been piloted in many countries worldwide and shown to improve both safety attitudes within surgical teams and patient outcomes. In the study...

  • Case Management and Outreach: Similarities and Differences in Worker Tasks. Macan, Therese; Cunningham, Jennifer; Lemming, Matthew R.; Calsyn, Robert J. // Care Management Journals;Summer2008, Vol. 9 Issue 2, p51 

    Case management and outreach are two important services provided to many individuals, particularly people with multiple problems. Unfortunately, no taxonomy or measuring instrument has been developed that captures the many dimensions of the jobs performed by both case managers and outreach...

  • Do You Like Music in Your OR? SHAPIRO, JARROD // Podiatry Management;Nov/Dec2011, Vol. 30 Issue 9, p43 

    The article presents an overview of the benefits that podiatrists and patients can find from having music playing in operating rooms while surgeries are taking place. In the article the author offers his opinions on the idea of having music playing in operating rooms while surgeries are taking...

  • The Effect of Clustering of Outcomes on the Association of Procedure Volume and Surgical Outcomes. Panageas, Katherine S.; Schrag, Deborah; Riedel, Elyn; Bach, Peter B.; Begg, Colin B. // Annals of Internal Medicine;10/21/2003, Vol. 139 Issue 8, p658 

    Assesses the effect of clustering of outcomes on the association of procedure volume and surgical outcomes of patients. Impact of available statistical techniques that correct for clustering; Effect of surgeon volume on hospital outcomes; Degree of clustering of outcomes.

  • Effectiveness of a physician-oriented feedback intervention on inappropriate hospital stays. Antón, Pedro; Peiró, Salvador; Aranaz, Jesos M.; Calpena, Rafael; Compañ, Antonio; Leutscher, Edith; Ruíz, Vicenta // Journal of Epidemiology & Community Health;Feb2007, Vol. 61 Issue 2, p128 

    Objective: To evaluate the effectiveness of a combined intervention to reduce inappropriate hospital stays. Design: Quasi-experimental pre-test/post-test with a non-equivalent control group. Setting: Three teaching hospitals in the National Health System in Alicante, Spain. Study participants:...

  • High volume and outcome after liver resection: surgeon or center? Eppsteiner, Robert W.; Csikesz, Nicholas G.; Simons, Jessica P.; Tseng, Jennifer F.; Shah, Shimul A. // Journal of Gastrointestinal Surgery;Oct2008, Vol. 12 Issue 10, p1709 

    Introduction: In a case controlled analysis, we attempted to determine if the volume-survival benefit persists in liver resection (LR) after eliminating differences in background characteristics.Methods: Using the Nationwide Inpatient Sample (NIS), we identified all LR...

  • Miracles. Rich, Cindy // Washingtonian Magazine;May2009, Vol. 44 Issue 8, p46 

    The article offers various stories of death, life and survival from the patients of transplant surgeon Lynt Johnson of Georgetown University Hospital. It mentions how Johnson was able to build a transplant program that help save lives of thousand people loosing hope of chances to survive their...

  • Trauma Team Activation: What do we learn? Komet Thongkhao; Burapat Sangthong; Osaree Akaraborworn // Thai Journal of Surgery;Jan-Mar2012, Vol. 33 Issue 1, p13 

    Background: The treatment outcome of severe trauma victims can be improved by the early presence of the trauma team under leadership of an expert trauma surgeon in the emergency room. The criteria for trauma team activation was the presence of patients at risk or need critical care. This study...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics