Abstract
Background
Visual representation help make the ever-increasing data more attractive, thought provoking and informative. In the field of surgical training, Procedure Based Assessment is a structured method of assessing surgical performance and skills of trainees in the UK and is a valuable tool for trainers in the Annual Review of Competence Progression. Trainers can view PBA's on the online-based Intercollegiate Surgical Curriculum Programme individually in a long-form format with no visual representation.
Aim
To assess the effect of an originally devised EVR tool of PBA's in the context of ARCP on 10 aspects including speed of assessment, assimilation of data, ease of interpretation and identification of trainees’ weaknesses and strengths.
Methodology
1) ISCP PBA data collected for three volunteered specialty trainees (ST4, ST5 & ST6) enrolled in Warwick Trauma and Orthopaedic training programme, for a six-month period from 1st July 2013 till 31st December 2013. 2) An EVR was generated using Tableau Desktop software, and two other EVR's originally devised to visually represent the trainees’ PBA's and integrated into three interactive PDF files. 3) Twelve trainer consultants participated in a mock ARCP and rated their experience in assessing the trainee's using the new EVR method compared to the ISCP website on three surveys. 4) A mock ARCP was set up for 12 consultants. To minimise bias, six assessors were randomised into two equal groups. Groups A were asked to use the ISCP website to formulate an ARCP decision and then use the EVR interactive tool. Group B used the EVR tool followed by browsing the ISCP website. 5) Assessors rated their experience after using each method and also at the end of the mock ARCP on three surveys. Responses recorded on a Visual Analogue Scale and statistically analysed using non-parametric a two-tailed Mann Whitney U test.
Results
Comparing responses to the EVR and ISCP surveys shows that users thought that using the EVR tool is more useful, accessible, easy to learn and use, time efficient and appealing. It also allowed them to better identify trainees’ areas of strengths and weaknesses and formulate a final ARCP outcome decision in relation to PBA's (p < 0.001). Strong agreement to develop the EVR tool and have incorporated into the current ISCP website have been demonstrated (p < 0.001). Comparison of total responses to the EVR and ISCP surveys between group A and group B showed no significant statistical difference
Conclusion
The project shows that Enhanced Visual Representation tool has the potential to be a positive addition to the ISCP website to improve the process of surgical training and feedback.
