Abstract
As patient data continues to grow, the importance of efficient and precise analysis cannot be overstated. The employment of Generative Artificial Intelligence (AI), specifically Chat GPT-4, in the realm of medical data interpretation has been on the rise. However, its effectiveness in comparison to manual data analysis has been insufficiently investigated.
This quality improvement project aimed to evaluate the accuracy and time-efficiency of Generative AI (GPT-4) against manual data interpretation within extensive datasets pertaining to patients with orthopaedic injuries.
A dataset, containing details of 6,562 orthopaedic trauma patients admitted to a district general hospital over a span of two years, was reviewed. Two researchers operated independently: one utilised GPT-4 for insights via prompts, while the other manually examined the identical dataset employing Microsoft Excel and IBM® SPSS® software. Both were blinded on each other's procedures and outcomes. Each researcher answered 20 questions based on the dataset including injury details, age groups, injury specifics, activity trends and the duration taken to assess the data.
Upon comparison, both GPT-4 and the manual researcher achieved consistent results for 19 out of the 20 questions (95% accuracy). After a subsequent review and refined prompts (prompt engineering) to GPT-4, the answer to the final question aligned with the manual researcher's findings. GPT-4 required just 30 minutes, a stark contrast to the manual researcher's 9-hour analytical duration.
This quality improvement project emphasises the transformative potential of Generative AI in the domain of medical data analysis. GPT-4 not only paralleled the accuracy of manual analysis but also achieved this in significantly less time. For optimal accurate results, data analysis by AI can be enhanced through human oversight. Adopting AI-driven approaches, particularly in orthopaedic data interpretation, can enhance efficiency and ultimately improve patient care. We recommend future investigations on large and more varied datasets to reaffirm these outcomes.
