Measuring intra-operative decision-making during laparoscopic cholecystectomy: validity evidence for a novel interactive Web-based assessment tool

Amin Madani, Yusuke Watanabe, Elif Bilgic, Philip H. Pucher, Melina C. Vassiliou, Rajesh Aggarwal, Gerald M. Fried, Elliot J. Mitmaker, Liane S. Feldman

Research output: Contribution to journalArticlepeer-review

21 Citations (Scopus)


Background: Errors in judgment during laparoscopic cholecystectomy can lead to bile duct injuries and other complications. Despite correlations between outcomes, expertise and advanced cognitive skills, current methods to evaluate these skills remain subjective, rater- and situation-dependent and non-systematic. The purpose of this study was to develop objective metrics using a Web-based platform and to obtain validity evidence for their assessment of decision-making during laparoscopic cholecystectomy. Methods: An interactive online learning platform was developed ( Trainees and surgeons from six institutions completed a 12-item assessment, developed based on a cognitive task analysis. Five items required subjects to draw their answer on the surgical field, and accuracy scores were calculated based on an algorithm derived from experts’ responses (“visual concordance test”, VCT). Test–retest reliability, internal consistency, and correlation with self-reported experience, Global Operative Assessment of Laparoscopic Skills (GOALS) score and Objective Performance Rating Scale (OPRS) score were calculated. Questionnaires were administered to evaluate the platform’s usability, feasibility and educational value. Results: Thirty-nine subjects (17 surgeons, 22 trainees) participated. There was high test–retest reliability (intraclass correlation coefficient = 0.95; n = 10) and internal consistency (Cronbach’s α = 0.87). The assessment demonstrated significant differences between novices, intermediates and experts in total score (p < 0.01) and VCT score (p < 0.01). There was high correlation between total case number and total score (ρ = 0.83, p < 0.01) and between total case number and VCT (ρ = 0.82, p < 0.01), and moderate to high correlations between total score and GOALS (ρ = 0.66, p = 0.05), VCT and GOALS (ρ = 0.83, p < 0.01), total score and OPRS (ρ = 0.67, p = 0.04), and VCT and OPRS (ρ = 0.78, p = 0.01). Most subjects agreed or strongly agreed that the platform and assessment was easy to use [n = 29 (78 %)], facilitates learning intra-operative decision-making [n = 28 (81 %)], and should be integrated into surgical training [n = 28 (76 %)]. Conclusion: This study provides preliminary validity evidence for a novel interactive platform to objectively assess decision-making during laparoscopic cholecystectomy.

Original languageEnglish
Pages (from-to)1203-1212
Number of pages10
JournalSurgical endoscopy
Issue number3
Publication statusPublished - 01-03-2017
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Surgery


Dive into the research topics of 'Measuring intra-operative decision-making during laparoscopic cholecystectomy: validity evidence for a novel interactive Web-based assessment tool'. Together they form a unique fingerprint.

Cite this