Purpose: Surgical trainees develop psychomotor skills using various techniques, with simulators providing safe practicing environments. There has been no development of virtual simulators with haptics (force feedback) that allow residents to practice the open surgical fixation of common orthopedic fractures. The main purpose of this study was to assess if residents performed similarly on a newly developed virtual simulator as on a Sawbones simulator using a modified checklist and global rating scale. Secondary purposes were to assess the reliability and validity of these procedural measurement tools.

Method: A stratified randomized within-subjects study was performed with 22 surgical trainee volunteers. They were randomized to first perform surgical fixation of the ulna using either the virtual or Sawbones simulator, and then performed the same procedure on the other simulator. Evaluators completed a task-specific checklist, global rating scale (GRS), total error score and time to completion for each participant on both simulators.

Results: The participants achieved significantly better scores on the virtual simulator compared to the Sawbones simulator (p0.8), except in time to completion. When combined, the checklist and GRS maintained high levels of internal consistency (Cronbach’s a > 0.80) and inter-rater reliability (intraclass coefficient > 0.90) for both simulators. A Pearson’s product moment correlation was used to demonstrate criterion validity of the measurement tools. They were all significantly correlated to each other within simulators (p0.9), while the virtual simulator achieved construct validity for the GRS and total error score (p1.1).

Conclusion: The modified procedural measurement tools demonstrate reliability and validity and the virtual simulator shows evidence of construct validity. These tools were used to evaluate participants, demonstrating the achievement of better scores on the virtual simulator compared to the Sawbones simulator. The only concern at this time is that the procedural measurement tool scores do not correlate between simulators. The newly developed virtual ulna surgical fixation simulator with haptics shows promise for helping surgical trainees learn and practice basic skills, but requires further modifications before it can attain the same standards as the current gold standard simulators.