External validation of machine learning predictive models is achieved through evaluation of model performance on different groups of patients than were used for algorithm development. This important step is uncommonly performed, inhibiting clinical translation of newly developed models. Recently, machine learning was used to develop a tool that can quantify revision risk for a patient undergoing primary anterior cruciate ligament (ACL) reconstruction (https://swastvedt.shinyapps.io/calculator_rev/). The source of data included nearly 25,000 patients with primary ACL reconstruction recorded in the Norwegian Knee Ligament Register (NKLR). The result was a well-calibrated tool capable of predicting revision risk one, two, and five years after primary ACL reconstruction with moderate accuracy. The purpose of this study was to determine the external validity of the NKLR model by assessing algorithm performance when applied to patients from the Danish Knee Ligament Registry (DKLR). The primary outcome measure of the NKLR model was probability of revision ACL reconstruction within 1, 2, and/or 5 years. For the index study, 24 total predictor variables in the NKLR were included and the models eliminated variables which did not significantly improve prediction ability - without sacrificing accuracy. The result was a well calibrated algorithm developed using the Cox Lasso model that only required five variables (out of the original 24) for outcome prediction. For this external validation study, all DKLR patients with complete data for the five variables required for NKLR prediction were included. The five variables were: graft choice, femur fixation device, Knee Injury and Osteoarthritis Outcome Score (KOOS) Quality of Life subscale score at surgery, years from injury to surgery, and age at surgery. Predicted revision probabilities were calculated for all DKLR patients. The model performance was assessed using the same metrics as the NKLR study: concordance and calibration. In total, 10,922 DKLR patients were included for analysis. Average follow-up time or time-to-revision was 8.4 (±4.3) years and overall revision rate was 6.9%. Surgical technique trends (i.e., graft choice and fixation devices) and injury characteristics (i.e., concomitant meniscus and cartilage pathology) were dissimilar between registries. The model produced similar concordance when applied to the DKLR population compared to the original NKLR test data (DKLR: 0.68; NKLR: 0.68-0.69). Calibration was poorer for the DKLR population at one and five years post primary surgery but similar to the NKLR at two years. The NKLR machine learning algorithm demonstrated similar performance when applied to patients from the DKLR, suggesting that it is valid for application outside of the initial patient population. This represents the first machine learning model for predicting revision ACL reconstruction that has been externally validated. Clinicians can use this in-clinic calculator to estimate revision risk at a patient specific level when discussing outcome expectations pre-operatively. While encouraging, it should be noted that the performance of the model on patients undergoing ACL reconstruction outside of Scandinavia remains unknown

Introduction. Dissatisfaction rates after TKA are reported to be between 15 – 25%, with unmet outcome expectations being a key contributor. Shared decision making tools (SDMT) are designed to align a patient's and surgeon's expectations. This study demonstrates clinical validation of a patient specific shared decision making tool. Methods. Patient reported outcome measures (PROMs) were collected in 150 patients in a pre-consultation environment of one surgeon. The data was processed into a probabilistic predictive model utilising prior data to generate a preoperative baseline and an expected outcome after TKA. The surgeon was blinded to the prediction algorithm for the first 75 patients and exposed for the following 75 patients. PROMs collected were the knee injury and osteoarthritis outcome score (KOOS) and questions on lower back pain, hip pain and falls. The patients booked and not booked before and after exposure to the prediction were collected. The clinical validation involved 27 patients who had their outcome predicted and had their PROMs captured at 12 months after TKA. The predicted change in severity of pain and the patients actual change from pre-op to 12 month post operative KOOS pain was analysed using a Spearman's Rho correlation. Further analysis was performed by dividing the group into those predicted by the model to have improved by more than 10 percentile points and those who were predicted to improve by less than 10 percentile points. Results. Prior to the clinical implementation of the application, the population of patients booked for TKR surgery had a preoperative KOOS pain score of 47.9 ± 17.1, while those not booked for TKR surgery had a mean KOOS pain score of 54.4 ± 21.0 points, with higher scores indicating a lower pain state. A difference of 6.5 points exists between the means. Following introduction of the application, the scores for the population of patients booked for TKR surgery were 40.0 ± 12.3, while those not booked were 55.2 ± 18.8, a significant difference of 15.2 (p<0.001). The clinical validation showed a strong correlation between the predicted and actual pain state change (Spearman's Rho = 0.63, p<0.0001). Patients who were predicted to have a change of less than 10 points pre- operatively had a lower KOOS total score at 12 months (72.16 vs 86.97, p = 0.02). Conclusions. We found a significant difference in the KOOS pain score of patients for whom a decision to operate was made following introduction of the application. A predictive algorithm based on PROMs may assist a surgeon to optimise their patient selection for TKR. The clinical validation showed a strong correlation between predicted and actual change in pain state before and after TKA, supporting the validity of the SDMT's prediction. Literature has shown that the change between pre TKA pain state and post TKA pain state influences patient satisfaction; those with a smaller change in reported pain being less satisfied. This concept has led to the development of a patient specific shared decision making tool that can be used by surgeons and patients in the pre TKA consultation