Abstract
Aims
The primary aim of the survey was to map the current provision of simulation training within UK and Republic of Ireland (RoI) trauma and orthopaedic (T&O) specialist training programmes to inform future design of a simulation based-curriculum. The secondary aims were to characterize; the types of simulation offered to trainees by stage of training, the sources of funding for simulation, the barriers to providing simulation in training, and to measure current research activity assessing the educational impact of simulation.
Methods
The development of the survey was a collaborative effort between the authors and the British Orthopaedic Association Simulation Group. The survey items were embedded in the Performance and Opportunity Dashboard, which annually audits quality in training across several domains on behalf of the Speciality Advisory Committee (SAC). The survey was sent via email to the 30 training programme directors in March 2019. Data were retrieved and analyzed at the Warwick Clinical Trials Unit, UK.
Results
Overall, 28 of 30 programme directors completed the survey (93%). 82% of programmes had access to high-fidelity simulation facilities such as cadaveric laboratories. More than half (54%) had access to a non-technical skills simulation training. Less than half (43%) received centralized funding for simulation, a third relied on local funding such as the departmental budget, and there was a heavy reliance on industry sponsorship to partly or wholly fund simulation training (64%). Provision was higher in the mid-stages (ST3-5) compared to late-stages (ST6-8) of training, and was formally timetabled in 68% of prostgrammes. There was no assessment of the impact of simulation training using objective behavioural measures or real-world clinical outcomes.
Conclusion
There is currently widespread, but variable, provision of simulation in T&O training in the UK and RoI, which is likely to expand further with the new curriculum. It is important that research activity into the impact of simulation training continues, to develop an evidence base to support investment in facilities and provision.
Introduction
Simulation has an increasingly important role in orthopaedic surgical training in a climate of reduced working hours,1,2 financial constraints3,4 and emphasis on patient safety.5 There is a move towards improving training efficiency and quality through curriculum reform to a competency-based model6 and the improved surgical training initiative,7 both of which prominently feature simulation.
The main appeal of simulation as a surgical training adjunct is that is moves the early part of the surgical learning curve away from patients8 into a controlled environment where competency can be measured and assured.9 It offers the potential for rapid ‘upskilling’, the expeditious attainment of surgical skill10 in a manner that is no longer achievable using the traditional master-apprentice model of ‘on-the-job’ training.
There are a wide variety of simulation training options available in orthopaedics, ranging from low fidelity bench-top box trainers11,12 and plastic bones13,14 for teaching basic orthopaedic skills, up to ultra-high fidelity simulation using human cadavers where an entire operation can be performed in a simulated operating theatre using real implants and instruments.15,16
The potential benefits of simulation also include the acquisition of non-technical skills, which forms a significant part of the skill set of a competent surgeon, the importance of which is only recently gaining recognition.17 There is an emerging body of evidence that both technical and non-technical skills learnt in the simulated environment can be transferred to the operating theatre,18,19 with potential benefit to patients.
Accordingly, there is a move to increase simulation provision within trauma and orthopaedics (T&O) training. It is therefore timely and necessary to provide a comprehensive overview of current simulation provision in T&O training programmes.
Methods
Survey development
The primary aim of the survey was to map the current provision of simulation training within UK and the Republic of Ireland (RoI) T&O specialist training programmes to inform future design of a simulation based-curriculum. The survey questions were developed to describe, by programme and region; the resources available for simulation, sources of funding, timetabling of provision within training, the type of simulation offered at each stage of training, and research activity measuring the educational impact of simulation training.
Participants and survey administration
The survey questions were embedded in the 2019 performance and opportunity dashboard data collection cycle, which annually audits quality of training in T&O across several domains on behalf of the speciality advisory committee (SAC). The questions were flagged as being for research with a brief outline of the study aims, and completion of this section was not mandatory for successful submission to the dashboard. The survey, hosted by a commercial web platform (Survey Monkey, San Mateo, California, USA), was sent by email to the 30 training programme directors (TPDs) in the UK and RoI in March 2019. Three email reminders were sent to non-responders and the survey was closed in July 2019.
Data Analysis
Data were retrieved by the SAC quality assurance lead (RJHG) and sent to the Warwick Clinical Trial Unit for analysis (HKJ). A descriptive analysis was undertaken, with all responses presented as counts and percentages. The results are presented visually as geothermal heat maps to show regional trends in provision ((Figs 1–6), respectively).
Fig. 1
Resources for simulation training by programme.
Fig. 2
Sources of funding for simulation training.
Fig. 3
Provision within training timetable and barriers to delivery.
Fig. 4
Provision by programme at the mid-stage of training (ST3-5).
Fig. 5
Provision by programme at the late stage of training (ST6-8).
Fig. 6
Research activity to measure the educational impact of simulation provision in training.
Results
Demographics
Overall, 28 out of 30 eligible TPDs completed the survey (93%). The geographical distribution of the 30 UK and ROI T&O training programmes are shown in Fig. 7. It should be noted that the geographical size of the programmes is independent of the number of trainees. The programme boundaries were determined by plotting the hospitals within a given rotation on Google maps (Google, Alphabet Inc, Mountain View, California, USA) and drawing boundaries around them. There are seven programmes in London, and considerable overlap between rotations among the London hospitals. A pragmatic, best-fit approach was therefore taken in determining the programme boundaries in London which allowed for a clear visual representation of the data.
Fig. 7
Geographical distribution of training programmes in the UK and RoI.
Resources for simulation delivery
Over 96% of programmes had facilities for simulated clinical case management, such as clinical skills teaching rooms within hospitals. Just over half of programmes (54%) had access to non-technical skills simulation, delivering modules related to situational awareness, decision making, communication and teamwork. In terms of technical surgical skills teaching, 82% had access to cadaver-based surgical procedure simulation and 64% of programmes had access to arthroscopy simulation. Less than one-third (29%) of programmes had access to a simulated operating theatre environment. It is probable that the fifteen programmes offering cadaveric simulation without access to a simulated operating theatre are delivering this training within dissection laboratories affiliated with medical schools, rather than dedicated surgical training centres.
Four programmes (14%, three within London) offer access to simulation facilities to trainees out-of-hours on an ‘as required’ basis.
Simulation resources by programme were classified as very low, low, moderate, high and very high according to the facilities available, and are shown in Fig. 1 with descriptors. There was no clear geographical pattern to provision of facilities for simulation, and there was considerable variation regionally. Training (on-line, on-paper, or face-to-face) was available for the simulation trainers in five regions (shown in Fig. 1 by the dotted shading), and this provision bore no relation to the overall quality of simulation facilities within the training programme. The opportunity to be involved in simulation training was made available to all trainers in just under half (46%) of programmes.
Sources of funding
Less than half (43%) of training programmes received centralised funding for simulation provision from Health Education England and/or the postgraduate deanery (Fig. 2). One-third (32%) of programmes funded simulation provision locally, using either NHS trust/departmental budget (two programmes), top-slicing trainee study budgets or trainee self-funding (six programmes) or via a charitable foundation (one programme). Of note, two-thirds (64%) of programmes received industry or other commercial sponsorship for the provision of simulation. There was no relationship between lack of centralized funding and reliance on industry sponsorship, and quantitative estimates of funding were not sought in the survey. The programmes with centralized funding support for simulation were mainly clustered in the south of England (Fig. 2). No funding information was given for six programmes.
Barriers to provision
The TPDs were asked about obstacles to provision of simulation within their respective programmes (Fig. 3). Overall, 40% of TPDs reported barriers to delivery of simulation; six programmes reported lack of facilities, two had funding issues, two reported logistical difficulties related to timetabling, one reported lack of available faculty, and one reported lack of trainee enthusiasm.
Formal provision of simulation within the timetable
Simulation was formally timetabled in 19 (68%) of training programmes, and was used as part of an enhanced induction programme in nine (32%). There was a clear relationship between the provision of a simulation enhanced induction programme and formal timetabling of simulation, with only one programme offering the former without the latter. Simulation was more likely to be formally provided in the timetable in England and Wales than in Scotland and Ireland/Northern Ireland (Fig. 3).
Type of simulation offered by stage of training
Provision of simulation by type and stage was measured, with mid (ST3-5) and late (ST6-8) specialist training considered separately. Low fidelity simulation was defined as box trainer or Sawbones (Sawbones Europe, Malmö, Sweden) or equivalent. Moderate fidelity was defined as virtual reality (VR) or animal tissue. High fidelity was defined as human cadaveric simulation, involving surgical approaches only, and ultra-high fidelity was the addition of implants, instruments and the ability to perform the entire procedure on the cadaver. Type of simulation offered by stage of training by programme is shown in Figs 4 and 5. As would be expected, the degree of fidelity offered within programmes broadly correlates with the facilities available for simulation (Fig. 1.) There is a trend towards less provision at the later stages of training, with two programmes offering no simulation provision at ST6-8, and three programmes offering high and ultra-high fidelity simulation to ST3-5 only.
Measuring the impact of simulation
Research activity measuring the educational impact of simulation provision was stratified according to Kirkpatrick’s hierarchy,20 which is a widely accepted framework for classifying the educational outcomes of a training intervention. Six programmes (21%) did not measure the impact of their simulation training. Six programmes (21%) measured the effect of simulation training on trainee opinion (Kirkpatrick level 1), which typically involves using pre- and post-training questionnaires to assess change in subjective metrics such as confidence. In all, 11 programmes (40%) assessed simulation provision using changes in trainee knowledge (Kirkpatrick level 2), through either bespoke post-training knowledge testing or the annual UKITE exercise. Neither of these methods assess technical or clinical skill improvement from simulation or transfer to the workplace. Two programmes in London (Fig. 6) report measuring the impact of simulation training using subjective behavioural measures (Kirkpatrick 3b). It is striking that there is no assessment of the impact of simulation within training programmes using the highest level of educational impact metrics - objective behavioural measures (Kirkpatrick level 3a) or patient outcomes (Kirkpatrick level 4).
Discussion
Orthopaedic educators face the considerable challenge of continuing to train high-quality surgeons amid increasing clinical, financial, regulatory and time pressures on training.21 The traditional master-apprentice model of surgical training, with its reliance on the two central tenets of high volume of case exposure and a sustained mentor-mentee relationship,22 has been rendered obsolete in the modern surgical healthcare environment.8 This is due largely to increased service demands at the expense of training,23 shift based working patterns, and short training rotations.
There is a growing evidence base supporting the use of simulation as an adjunct to training, showing that the learning curve can be advanced away from patients,10 with inherent safety advantages,24 and that skills learnt in the simulated environment can transfer to the operating theatre.19 There have been many studies demonstrating the face25 and construct26,27 validity, feasibility28 and educational impact29,30 of various orthopaedic simulators for both open and arthroscopic surgery, ranging from low-fidelity, low-cost box-trainer type models through to ultra-high-fidelity cadaveric simulation.
This study is the first to map simulation in T&O training, and describe its current provision status on a national scale. The results of the study show that overall, simulation provision is highly variable across the 30 T&O training programmes of the UK and RoI. The availability of resources varies widely (Fig. 1), and is likely to be at least partly influenced by the geographical and financial relationship of training programmes with University medical schools, where cadaveric wet-laboratory facilities are generally situated.
The funding landscape for simulation provision is complex and not explored in detail in this study. There was a tendency for simulation provision in the southern half of England to be described as centrally funded, the reason for which is unclear, and a widespread reliance on industry sponsorship (64%) was seen, seemingly independent of centralized funding status. Of note, four programmes reported that trainees were required to self-fund simulation training. This raises obvious ethical concerns around equity of access to training opportunity, as some trainees may be disadvantaged through being unable to pay to access simulation training. Six studies reported funding simulation training by top-slicing trainees’ study budget, which although fairer than asking trainees to self-fund, erodes their already limited study budget and may jeopardise access to other valuable training opportunities such as attendance at courses and conferences.
Barriers to formally integrating simulation in the timetable were explored. Barriers cited included; lack of facilities (n = 6), lack of faculty (n = 1), funding issues (n = 2), logistical issues with timetabling (n = 2) and, surprisingly, lack of trainee enthusiasm (n = 1). The majority of programmes (68%) have formally timetabled simulation provision, which is important for logistical considerations when planning training, and also assures the trainees of protected ‘bleep-free’ teaching time. Formal timetabling also removes the need for a separate study leave application process or annual leave usage to attend simulation training, and attendance rates can be monitored.
Type of simulation offered by stage of training and region is shown in Figs 4 and 5. A disparity is seen between available facilities and reported provision in Glasgow, where ultra-high fidelity training is provided at both the mid- and late-stages of training but the resources for simulation is given as very low with no cadaveric facilities. One explanation is that cadaveric training is outsourced to another region, which we did not address in the survey. Less simulation provision was generally available during late stage training (ST6-8). This may be explained by the potentially greatest gains being obtained from simulation at an earlier stage of training, where the learning curve is steeper.
A limitation of this study is that we have focussed on establishing facts about provision at the expense of a more nuanced understanding of TPD opinion on the role of simulation in their training programmes. We have also not included trainees in this study, who as the direct beneficiaries of simulation training, should be central to discussions around provision. We obtained a very high response rate of 93% but the survey cannot claim to be comprehensive as there are two training programmes for which we have no data. Provision of simulation is only one of several quality indicators relevant to assessing a training programme, and we make no inferences as to the quality of individual training programmes based on these results.
Despite the obvious appeal of simulation as a part-solution to the challenges of the modern surgical training environment, there is no evidence to date that simulation training in orthopaedics benefits patients (Kirkpatrick level 4 evidence). Orthopaedics lags behind general surgery in its efforts to measure the educational impact of simulation training, in a systematic review18 of skill transfer to the operating theatre after simulation training only one31 of 34 studies was from orthopaedics. Research efforts to measure so called ‘transfer validity’ have been complicated by a lack of appropriately validated objective outcome measures,32 and are largely restricted to the research setting. Only two studies31,33 have shown evidence of trasfer validity following simulation training in orthopaedics, both of these involve diagnostic knee arthroscopy. Arthroscopic procedures lend themselves more easily to objective measurement of skill transfer, as motion analysis can be used to objectively measure performance in the simulated environment, and subsequently the operating theatre. The measurement of transfer validity of open procedures is considerably harder.32
Simulation delivery can be costly, cadaveric training especially so.34 Until such a time that there is a high-quality evidence base showing that simulation training in T&O improves technical and non-technical skills that translate into the workplace for the benefit of patients, it is unlikely that simulation is going to be mandated for training or comprehensively funded by HEE.
The survey results show that despite some challenges there is currently widespread simulation provision in training, much of it using sophisticated techniques such as cadaveric simulation (Figs 4 and 5). The use of innovative funding streams, and widespread efforts, albeit imperfect, to measure the educational benefit of simulation suggests a high level of enthusiasm for the delivery of simulation in T&O. These results reveal a promising foundation for the future of simulation delivery in T&O training, which we anticipate will continue to grow further with the implementation of the new curriculum next year. It is important that research activity continues keeps pace with the anticipated expansion of simulation provision, and can inform future developments in an evidence based manner.
References
1. Pickersgill T . The European working time directive for doctors in training . BMJ . 2001 ; 323 ( 7324 ): 1266 . Crossref PubMed Google Scholar
2. House J . Calling time on doctors' working hours . The Lancet . 2009 ; 373 ( 9680 ): 2011 – 2012 . Crossref PubMed Google Scholar
3. The state of medical education and practice in the UK . London : General Medical Council , 2016 . Google Scholar
4. Department of Health - Third Report . House of commons, editor . London : The Stationary Office , 2008 . Google Scholar
5. Gaskell N , Hinton R , Page T , Elvins T , Malin A . Putting an end to Black Wednesday: improving patient safety by achieving comprehensive trust induction and mandatory training by day 1 . Clin Med . 2016 ; 16 ( 2 ): 124 – 128 . Crossref PubMed Google Scholar
6. Bowditch MG . Editor curriculum update. British orthopaedic association annual Congress . Liverpool , 2019 . Google Scholar
7. Improving Surgical Training . Proposal for a pilot surgical training programme . 2015 . Google Scholar
8. Reznick RK , MacRae H . Teaching surgical skills — changes in the wind . N Engl J Med . 2006 ; 355 ( 25 ): 2664 – 2669 . Google Scholar
9. MacCaskie A , Kenny D , Deshmukh S . How can surgical training benefit from theories of skilled motor development . musical skill acquisition and performance psychology? The Medical Journal of Australia . 2011 ; 194 ( 9 ). Crossref PubMed Google Scholar
10. Curran I . A framework for technology enhanced learning . London : Department of Health , 2011 . Google Scholar
11. Hohn EA , Brooks AG , Leasure J , et al. Development of a surgical skills curriculum for the training and assessment of manual skills in orthopedic surgical residents . J Surg Educ . 2015 ; 72 ( 1 ): 47 – 52 . Crossref PubMed Google Scholar
12. Ruder JA , Turvey B , Hsu JR , Scannell BP . Effectiveness of a low-cost drilling module in orthopaedic surgical simulation . J Surg Educ . 2017 ; 74 ( 3 ): 471 – 476 . Crossref PubMed Google Scholar
13. LeBlanc J , Hutchison C , Hu Y , Donnon T . A comparison of orthopaedic resident performance on surgical fixation of an ulnar fracture using virtual reality and synthetic models . The Journal of Bone & Joint Surgery . 2013 ; 95 ( 9 ): e60 – 65 . Crossref PubMed Google Scholar
14. Leong JJH , Leff DR , Das A , et al. Validation of orthopaedic bench models for trauma surgery . J Bone Joint Surg Br . 2008 ; 90-B ( 7 ): 958 – 965 . Crossref PubMed Google Scholar
15. Gottschalk MB , Yoon ST , Park DK , Rhee JM , Mitchell PM . Surgical training using three-dimensional simulation in placement of cervical lateral mass screws: a blinded randomized control trial . The Spine Journal . 2015 ; 15 ( 1 ): 168 – 175 . Crossref PubMed Google Scholar
16. Camp CL , Krych AJ , Stuart MJ , et al. Improving resident performance in knee arthroscopy: a prospective value assessment of simulators and cadaveric skills laboratories . Journal of Bone and Joint Surgery - American Volume . 2016 ; 98 ( 3 ): 220 – 225 . Crossref PubMed Google Scholar
17. Agha RA , Fowler AJ , Sevdalis N . The role of non-technical skills in surgery . Annals of Medicine and Surgery . 2015 ; 4 ( 4 ): 422 – 427 . Crossref PubMed Google Scholar
18. Dawe SR , Pena GN , Windsor JA , et al. Systematic review of skills transfer after surgical simulation-based training . Br J Surg . 2014 ; 101 ( 9 ): 1063 – 1076 . Crossref PubMed Google Scholar
19. Sturm LP , Windsor JA , Cosman PH , et al. A systematic review of skills transfer after surgical simulation training . Ann Surg . 2008 ; 248 ( 2 ): 166 – 179 . Crossref PubMed Google Scholar
20. Kirkpatrick D . Techniques for evaluating training programmes . Journal of American Society for Training and Development . 1959 ; 13 ( 11 ): 11 – 12 . Google Scholar
21. Karam MD , Pedowitz RA , Natividad H , Murray J , Marsh JL . Current and future use of surgical skills training laboratories in orthopaedic resident education: a national survey . J Bone Joint Surg Am . 2013 ; 95 ( 1 ): e4. Crossref PubMed Google Scholar
22. Walter AJ . Surgical education for the twenty-first century: beyond the apprentice model . Obstet Gynecol Clin North Am . 2006 ; 33 ( 2 ): 233 – 236 . Crossref PubMed Google Scholar
23. James HK , Fisher JG , Griffin DR , Pattison G . If we can’t get to theatre, we can’t learn to operate: A qualitative study of factors influencing core trainee access to the operating theatre in Trauma & Orthopaedics . Ann R Coll Surg Engl . In Press . 2019 . Google Scholar
24. Scott DJ , Safety P . Patient safety, competency, and the future of surgical simulation . Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare . 2006 ; 1 ( 3 ): 164 – 170 . Crossref PubMed Google Scholar
25. Tuijthof GJM , van Sterkenburg MN , Sierevelt IN , et al. First validation of the PASSPORT training environment for arthroscopic skills. knee surgery, sports Traumatology . Arthroscopy . 2010 ; 18 ( 2 ): 218 – 224 . Google Scholar
26. Angelo RL , Ryu RKN , Pedowitz RA , Gallagher AG . Metric development for an arthroscopic Bankart procedure: assessment of face and content validity . Arthroscopy: The Journal of Arthroscopic & Related Surgery . 2015 ; 31 ( 8 ): 1430 – 1440 . Crossref PubMed Google Scholar
27. Colaco HB , Hughes K , Pearse E , Arnander M , Tennent D . Construct validity, assessment of the learning curve, and experience of using a low-cost arthroscopic surgical simulator . J Surg Educ . 2017 ; 74 ( 1 ): 47 – 54 . Crossref PubMed Google Scholar
28. Coughlin RP , Pauyo T , Sutton JC , Coughlin LP , Bergeron SG . A validated orthopaedic surgical simulation model for training and evaluation of basic arthroscopic skills . J Bone Joint Surg Am . 2015 ; 97 ( 17 ): 1465 – 1471 . Crossref PubMed Google Scholar
29. Cannon WD , Nicandri GT , Reinig K , Mevis H , Wittstein J . Evaluation of skill level between trainees and community orthopaedic surgeons using a virtual reality arthroscopic knee simulator . J Bone Joint Surg Am . 2014 ; 96 ( 7 ): e57. Crossref PubMed Google Scholar
30. Rebolledo BJ , Hammann-Scala J , Leali A , Ranawat AS . Arthroscopy skills development with a surgical simulator: a comparative study in orthopaedic surgery residents . The American journal of sports medicine . 2015 ; 43 ( 6 ): 1526 – 1529 . Crossref PubMed Google Scholar
31. Howells NR , Gill HS , Carr AJ , Price AJ , Rees JL . Transferring simulated arthroscopic skills to the operating theatre . J Bone Joint Surg Br . 2008 ; 90-B ( 4 ): 494 – 499 . Crossref PubMed Google Scholar
32. James HK . Measuring the educational impact of simulation training in Trauma & Orthopaedics . Journal of Trauma & Orthopaedics . In Press . 2019 . Google Scholar
33. Garfjeld Roberts P , Alvand A , Gallieri M , Hargrove C , Rees J . Objectively assessing intraoperative arthroscopic skills performance and the transfer of simulation training in knee arthroscopy: a randomized controlled trial . Arthroscopy: The Journal of Arthroscopic & Related Surgery . 2019 ; 35 ( 4 ): 1197 – 1209 . Crossref PubMed Google Scholar
34. Gilbody J , Prasthofer AW , Ho K , Costa ML . The use and effectiveness of cadaveric workshops in higher surgical training: a systematic review . Annals . 2011 ; 93 ( 5 ): 347 – 352 . Crossref PubMed Google Scholar
Funding statement
H. K. James holds a Versus Arthritis Educational Research Fellowship (grant number 20485).
Follow H. James @hannah_ortho
© 2020 Author(s) et al. This is an open-access article distributed under the terms of the Creative Commons Attributions licence (CC-BY-NC-ND), which permits unrestricted use, distribution, and reproduction in any medium, but not for commercial gain, provided the original author and source are credited. Standard
