External validation of machine learning predictive models is achieved through evaluation of model performance on different groups of patients than were used for algorithm development. This important step is uncommonly performed, inhibiting clinical translation of newly developed models. Recently, machine learning was used to develop a tool that can quantify revision risk for a patient undergoing primary anterior cruciate ligament (ACL) reconstruction (https://swastvedt.shinyapps.io/calculator_rev/). The source of data included nearly 25,000 patients with primary ACL reconstruction recorded in the Norwegian Knee Ligament Register (NKLR). The result was a well-calibrated tool capable of predicting revision risk one, two, and five years after primary ACL reconstruction with moderate accuracy. The purpose of this study was to determine the external validity of the NKLR model by assessing algorithm performance when applied to patients from the Danish Knee Ligament Registry (DKLR). The primary outcome measure of the NKLR model was probability of revision ACL reconstruction within 1, 2, and/or 5 years. For the index study, 24 total predictor variables in the NKLR were included and the models eliminated variables which did not significantly improve prediction ability - without sacrificing accuracy. The result was a well calibrated algorithm developed using the Cox Lasso model that only required five variables (out of the original 24) for outcome prediction. For this external validation study, all DKLR patients with complete data for the five variables required for NKLR prediction were included. The five variables were: graft choice, femur fixation device, Knee Injury and Osteoarthritis Outcome Score (KOOS) Quality of Life subscale score at surgery, years from injury to surgery, and age at surgery. Predicted revision probabilities were calculated for all DKLR patients. The model performance was assessed using the same metrics as the NKLR study: concordance and calibration. In total, 10,922 DKLR patients were included for analysis. Average follow-up time or time-to-revision was 8.4 (±4.3) years and overall revision rate was 6.9%. Surgical technique trends (i.e., graft choice and fixation devices) and injury characteristics (i.e., concomitant meniscus and cartilage pathology) were dissimilar between registries. The model produced similar concordance when applied to the DKLR population compared to the original NKLR test data (DKLR: 0.68; NKLR: 0.68-0.69). Calibration was poorer for the DKLR population at one and five years post primary surgery but similar to the NKLR at two years. The NKLR machine learning algorithm demonstrated similar performance when applied to patients from the DKLR, suggesting that it is valid for application outside of the initial patient population. This represents the first machine learning model for predicting revision ACL reconstruction that has been externally validated. Clinicians can use this in-clinic calculator to estimate revision risk at a patient specific level when discussing outcome expectations pre-operatively. While encouraging, it should be noted that the performance of the model on patients undergoing ACL reconstruction outside of Scandinavia remains unknown

Aim. This study aimed to externally validate promising preoperative PJI prediction models in a recent, multinational European cohort. Method. Three preoperative PJI prediction models (by Tan et al., Del Toro et al., and Bülow et al.) which previously demonstrated high levels of accuracy were selected for validation. A multicenter retrospective observational analysis was performed of patients undergoing total hip arthroplasty (THA) and total knee arthroplasty (TKA) between January 2020 and December 2021 and treated at centers in the Netherlands, Portugal, and Spain. Patient characteristics were compared between our cohort and those used to develop the prediction models. Model performance was assessed through discrimination and calibration. Results. A total of 2684 patients were included of whom 60 developed a PJI (2.2%). Our patient cohort differed from the models’ original cohorts in terms of demographic variables, procedural variables, and the prevalence of comorbidities. The c-statistics for the Tan, Del Toro, and Bülow models were 0.72, 0.69, and 0.72 respectively. Calibration was reasonable, but precise percentage estimates for PJI risk were most accurate for predicted risks up to 3-4%; the Tan model overestimated risks above 4%, while the Del Toro model underestimated risks above 3%. Conclusions. In this multinational cohort study, the Tan, Del Toro, and Bülow PJI prediction models were found to be externally valid for classifying high risk patients for developing a PJI. These models hold promise for clinical application to enhance preoperative patient counseling and targeted prevention strategies. Keywords. Periprosthetic Joint Infection (PJI), High Risk Groups, Prediction Models, Validation, Infection Prevention

Advances in cancer therapy have prolonged patient survival even in the presence of disseminated disease and an increasing number of cancer patients are living with metastatic bone disease (MBD). The proximal femur is the most common long bone involved in MBD and pathologic fractures of the femur are associated with significant morbidity, mortality and loss of quality of life (QoL). Successful prophylactic surgery for an impending fracture of the proximal femur has been shown in multiple cohort studies to result in longer survival, preserved mobility, lower transfusion rates and shorter post-operative hospital stays. However, there is currently no optimal method to predict a pathologic fracture. The most well-known tool is Mirel's criteria, established in 1989 and is limited from guiding clinical practice due to poor specificity and sensitivity. The ideal clinical decision support tool will be of the highest sensitivity and specificity, non-invasive, generalizable to all patients, and not a burden on hospital resources or the patient's time. Our research uses novel machine learning techniques to develop a model to fill this considerable gap in the treatment pathway of MBD of the femur. The goal of our study is to train a convolutional neural network (CNN) to predict fracture risk when metastatic bone disease is present in the proximal femur. Our fracture risk prediction tool was developed by analysis of prospectively collected data of consecutive MBD patients presenting from 2009–2016. Patients with primary bone tumors, pathologic fractures at initial presentation, and hematologic malignancies were excluded. A total of 546 patients comprising 114 pathologic fractures were included. Every patient had at least one Anterior-Posterior X-ray and clinical data including patient demographics, Mirel's criteria, tumor biology, all previous radiation and chemotherapy received, multiple pain and function scores, medications and time to fracture or time to death. We have trained a convolutional neural network (CNN) with AP X-ray images of 546 patients with metastatic bone disease of the proximal femur. The digital X-ray data is converted into a matrix representing the color information at each pixel. Our CNN contains five convolutional layers, a fully connected layers of 512 units and a final output layer. As the information passes through successive levels of the network, higher level features are abstracted from the data. The model converges on two fully connected deep neural network layers that output the risk of fracture. This prediction is compared to the true outcome, and any errors are back-propagated through the network to accordingly adjust the weights between connections, until overall prediction accuracy is optimized. Methods to improve learning included using stochastic gradient descent with a learning rate of 0.01 and a momentum rate of 0.9. We used average classification accuracy and the average F1 score across five test sets to measure model performance. We compute F1 = 2 x (precision x recall)/(precision + recall). F1 is a measure of a model's accuracy in binary classification, in our case, whether a lesion would result in pathologic fracture or not. Our model achieved 88.2% accuracy in predicting fracture risk across five-fold cross validation testing. The F1 statistic is 0.87. This is the first reported application of convolutional neural networks, a machine learning algorithm, to this important Orthopaedic problem. Our neural network model was able to achieve reasonable accuracy in classifying fracture risk of metastatic proximal femur lesions from analysis of X-rays and clinical information. Our future work will aim to externally validate this algorithm on an international cohort

Technology within medicine has great potential to bring about more accessible, efficient, and a higher quality delivery of care. Paediatric supracondylar fractures are the most common elbow fracture in children and at our institution often have high rates of unnecessary long term clinical follow-up, leading to an inefficient use of healthcare and patient resources. This study aims to evaluate patient and clinical factors that significantly predict necessity for further clinical visits following closed reduction and percutaneous pinning. A total of 246 children who underwent closed reduction and percutaneous pinning following supracondylar humerus fractures were prospectively enrolled over a two year period. Patient demographics, perioperative course, goniometric measurements, functional outcome measures, clinical assessment and decision making for further follow up were assessed. Categorical and continuous variables were analyzed and screened for significance via bivariate regression. Significant covariates were used to develop a predictive model through multivariate logistical regression. A probability cut-off was determined on the Receiver Operator Characteristic (ROC) curve using the Youden index to maximize sensitivity and specificity. The regression model performance was then prospectively tested against 22 patients in a blind comparison to evaluate accuracy. 246 paediatrics patients were collected, with 29 cases requiring further follow up past the three month visit. Significant predictive factors for follow up were residual nerve palsy (p < 0 .001) and maximum active flexion angle of injured elbow (p < 0 .001). Insignificant factors included other goniometric measures, subjective evaluations, and functional outcomes scores. The probability of requiring further clinical follow up at the 3 month post-op point can be estimated with the equation: logit(follow-up) = 11.319 + 5.518(nerve palsy) − 0.108(maximum active flexion). Goodness of fit of the model was verified with Nagelkerke R2 = 0.574 and Hosmer & Lemeshow chi-square (p = 0.739). Area Under Curve of the ROC curve was C = 0.919 (SE = 0.035, 95% CI 0.850 – 0.988). Using Youden's Index, a cut-off for probability of follow up was set at 0.094 with the overall sensitivity and specificity maximized to 86.2% and 88% respectively. Using this model and cohort, 194 three month clinic visits would have been deemed medically unnecessary. Preliminary blind prospective testing against the 22 patient cohort demonstrates a model sensitivity and specificity at 100% and 75% respectively, correctly deeming 15 visits unnecessary. Virtual clinics and automated clinical decision making can improve healthcare inefficiencies, unclog clinic wait times, and ultimately enhance quality of care delivery. Our regression model is highly accurate in determining medical necessity for physician examination at the three month visit following supracondylar fracture closed reduction and percutaneous pinning. When applied correctly, there is potential for significant reductions in health care expenditures and in the economic burden on patient families by removing unnecessary visits. In light of positive patient and family receptiveness toward technology, our promising findings and predictive model may pave the way for remote health care delivery, virtual clinics, and automated clinical decision making

Advances in cancer therapy have prolonged cancer patient survival even in the presence of disseminated disease and an increasing number of cancer patients are living with metastatic bone disease (MBD). The proximal femur is the most common long bone involved in MBD and pathologic fractures of the femur are associated with significant morbidity, mortality and loss of quality of life (QoL). Successful prophylactic surgery for an impending fracture of the proximal femur has been shown in multiple cohort studies to result in patients more likely to walk after surgery, longer survival, lower transfusion rates and shorter post-operative hospital stays. However, there is currently no optimal method to predict a pathologic fracture. The most well-known tool is Mirel's criteria, established in 1989 and is limited from guiding clinical practice due to poor specificity and sensitivity. The goal of our study is to train a convolutional neural network (CNN) to predict fracture risk when metastatic bone disease is present in the proximal femur. Our fracture risk prediction tool was developed by analysis of prospectively collected data for MBD patients (2009–2016) in order to determine which features are most commonly associated with fracture. Patients with primary bone tumors, pathologic fractures at initial presentation, and hematologic malignancies were excluded. A total of 1146 patients comprising 224 pathologic fractures were included. Every patient had at least one Anterior-Posterior X-ray. The clinical data includes patient demographics, tumor biology, all previous radiation and chemotherapy received, multiple pain and function scores, medications and time to fracture or time to death. Each of Mirel's criteria has been further subdivided and recorded for each lesion. We have trained a convolutional neural network (CNN) with X-ray images of 1146 patients with metastatic bone disease of the proximal femur. The digital X-ray data is converted into a matrix representing the color information at each pixel. Our CNN contains five convolutional layers, a fully connected layers of 512 units and a final output layer. As the information passes through successive levels of the network, higher level features are abstracted from the data. This model converges on two fully connected deep neural network layers that output the fracture risk. This prediction is compared to the true outcome, and any errors are back-propagated through the network to accordingly adjust the weights between connections. Methods to improve learning included using stochastic gradient descent with a learning rate of 0.01 and a momentum rate of 0.9. We used average classification accuracy and the average F1 score across test sets to measure model performance. We compute F1 = 2 x (precision x recall)/(precision + recall). F1 is a measure of a test's accuracy in binary classification, in our case, whether a lesion would result in pathologic fracture or not. Five-fold cross validation testing of our fully trained model revealed accurate classification for 88.2% of patients with metastatic bone disease of the proximal femur. The F1 statistic is 0.87. This represents a 24% error reduction from using Mirel's criteria alone to classify the risk of fracture in this cohort. This is the first reported application of convolutional neural networks, a machine learning algorithm, to an important Orthopaedic problem. Our neural network model was able to achieve impressive accuracy in classifying fracture risk of metastatic proximal femur lesions from analysis of X-rays and clinical information. Our future work will aim to validate this algorithm on an external cohort

This is quite an innovative study that should lead to a multicentre validation trial. We have developed an FDG-PET/MRI texture-based model for the prediction of lung metastases (LM) in newly diagnosed patients with soft-tissue sarcomas (STSs) using retrospective analysis. In this work, we assess the model performance using a new prospective STS cohort. We also investigate whether incorporating hypoxia and perfusion biomarkers derived from FMISO-PET and DCE-MRI scans can further enhance the predictive power of the model. A total of 66 patients with histologically confirmed STSs were used in this study and divided into two groups: a retrospective cohort of 51 patients (19 LM) used for training the model, and a prospective cohort of 15 patients (two patients with LM, one patient with bone metastases and suspicious lung nodules) for testing the model. In the training phase, a model of four texture features characterising tumour sub-region size and intensity heterogeneities was developed for LM prediction from pre-treatment FDG-PET and MRI scans (T1-weighted, T2-weighted with fat saturation) of the retrospective cohort, using imbalance-adjusted bootstrap statistical resampling and logistic regression multivariable modeling. In the testing phase, this multivariable model was applied to predict the distant metastasis status of the prospective cohort. The predictive power of the obtained model response was assessed using the area under the receiver-operating characteristic curve (AUC). In the exploratory phase of the study, we extracted two heterogeneity metrics from the prospective cohort: the area under the intensity-volume histogram of pre-treatment DCE-MRI volume transfer constant parametric maps and FMISO-PET hypoxia maps (AU-IVH-Ktrans, AU-IVH-FMISO). The impact of the addition of these two individual metrics to the texture-based model response obtained in the testing phase was first investigated using Spearman's correlation (rs), and lastly using logistic regression and leave-one-out cross-validation (LOO-CV) to account for overfitting bias. First, the texture-based model reached an AUC of 0.94, a sensitivity of 1, a specificity of 0.83 and an accuracy of 0.87 when tested in the prospective cohort. In the exploratory phase, the addition of AU-IVH-FMISO did not improve predictive power, yielding a correlation of rs = −0.42 (p = 0.12) with lung metastases, and a relative change in validation AUC of 0% in comparison with the texture-based model response alone in LOO-CV experiments. In contrast, the addition of AU-IVH-Ktrans improved predictive power, yielding a correlation of rs = −0.54 (p = 0.04) with lung metastases, and a change in validation AUC of +10%. Our results demonstrate that texture-based models extracted from pre-treatment FDG-PET and MRI anatomical scans could be successfully used to predict distant metastases in STS cancer. Our results also suggest that the addition of perfusion heterogeneity metrics may contribute to improving model prediction performance

Background. The advent of value-based conscientiousness and rapid-recovery discharge pathways presents surgeons, hospitals, and payers with the challenge of providing the same total hip arthroplasty episode of care in the safest and most economic fashion for the same fee, despite patient differences. Various predictive analytic techniques have been applied to medical risk models, such as sepsis risk scores, but none have been applied or validated to the elective primary total hip arthroplasty (THA) setting for key payment-based metrics. The objective of this study was to develop and validate a predictive machine learning model using preoperative patient demographics for length of stay (LOS) after primary THA as the first step in identifying a patient-specific payment model (PSPM). Methods. Using 229,945 patients undergoing primary THA for osteoarthritis from an administrative database between 2009– 16, we created a naïve Bayesian model to forecast LOS after primary THA using a 3:2 split in which 60% of the available patient data “built” the algorithm and the remaining 40% of patients were used for “testing.” This process was iterated five times for algorithm refinement, and model performance was determined using the area under the receiver operating characteristic curve (AUC), percent accuracy, and positive predictive value. LOS was either grouped as 1–5 days or greater than 5 days. Results. The machine learning model algorithm required age, race, gender, and two comorbidity scores (“risk of illness” and “risk of morbidity”) to demonstrate excellent validity, reliability, and responsiveness with an AUC of 0.87 after five iterations. Hospital stays of greater than 5 days for THA were most associated with increased risk of illness and risk of comorbidity scores during admission compared to 1–5 days of stay. Conclusions. Our machine learning model derived from administrative big data demonstrated excellent validity, reliability, and responsiveness after primary THA while accurately predicting LOS and identifying two comorbidity scores as key value-based metrics. Predictive data has the potential to engender a risk-based PSPM prior to primary THA and other elective orthopaedic procedures

Aims

Disorders of bone integrity carry a high global disease burden, frequently requiring intervention, but there is a paucity of methods capable of noninvasive real-time assessment. Here we show that miniaturized handheld near-infrared spectroscopy (NIRS) scans, operated via a smartphone, can assess structural human bone properties in under three seconds.

Despite the vast quantities of published artificial intelligence (AI) algorithms that target trauma and orthopaedic applications, very few progress to inform clinical practice. One key reason for this is the lack of a clear pathway from development to deployment. In order to assist with this process, we have developed the Clinical Practice Integration of Artificial Intelligence (CPI-AI) framework – a five-stage approach to the clinical practice adoption of AI in the setting of trauma and orthopaedics, based on the IDEAL principles (https://www.ideal-collaboration.net/). Adherence to the framework would provide a robust evidence-based mechanism for developing trust in AI applications, where the underlying algorithms are unlikely to be fully understood by clinical teams.

Cite this article: Bone Joint Res 2024;13(9):507–512.

Joint assessment through manual physical examination is a fundamental skill that must be acquired by orthopaedic surgeons. These joint assessments allow surgeons to identify soft tissue injuries (e.g. ligament tears) which are critical in identifying appropriate treatment options. The difficulty in communicating the feeling of different joint conditions and the limited opportunities for practice can make these skills challenging to learn, resulting in reduced treatment effectiveness and increased costs. This research seeks to improve the training of joint assessment with the creation of a haptic joint simulator that can train surgeons with increased effectiveness. A first of its kind haptic simulator is presented, which incorporates: a newly defined kinetic knee simulation, a haptic device for user interaction, and a haptic control algorithm. The knee model has been specifically created for this application and allows six degree-of-freedom manipulation of the tibia while considering the effects of ten knee ligament bundles. The model has been mathematically formulated to allow for the high update rates necessary for smooth and stable haptic simulation. Two quantitative assessments were made of the model to confirm its clinical validity. The first was against the widely used OpenSim biomechanical simulation software. Simulations of the model's performance for both anterior-posterior draw tests and varus-valgus rotation tests showed less than 0.7%RMSE for force and 5.5%RMSE for moments. Crucially, the proposed model could generate updated forces in less than 1ms, compared to 188ms for OpenSim. The second validation of the model was against a cadaveric knee that was tested using a validated robotic testing platform. This comparison showed that the model could generate similar force- motion pathways to the cadaveric knee after the model's parameters were scaled to match. Having demonstrated that it is possible to create a computational knee model that has good conformance to gold-standard knee simulations and cadaveric recordings, while updating at less than 1ms, this research has overcome a major hurdle. The next stage of this research will be to incorporate the knee model into a full haptic simulator and perform skill acquisition trials. Given the effectiveness of past haptic training systems in aiding clinical skills acquisition, this research offers a promising way to improve surgeon training, and therefore also patient diagnosis and treatment

Aims

The use of technology to assess balance and alignment during total knee surgery can provide an overload of numerical data to the surgeon. Meanwhile, this quantification holds the potential to clarify and guide the surgeon through the surgical decision process when selecting the appropriate bone recut or soft tissue adjustment when balancing a total knee. Therefore, this paper evaluates the potential of deploying supervised machine learning (ML) models to select a surgical correction based on patient-specific intra-operative assessments.