Abstract
INTRODUCTION
The timely identification of outliers (implants, surgeons or patients) using prospectively collected registry data is confounded by many factors, including the assumption that the sampled population is representative of the entire cohort of patients. In this study we utilized a computer simulation of a joint registry to address the question: How does incomplete enrollment of patients in registries affect the reliability of identification of outliers, and what percent capture of the target population is sufficient?
MATERIALS AND METHODS
A synthetic registry was created consisting of 10,000 patients (100 surgeons), of whom, 1000 underwent joint replacement using a new implant. A predictive model for the risk of revision was created from data published by the Swedish TKR Registry and the AOANJRR. The pairing of patients, surgeons and implants was randomized and for each assignment, the probability of revision was computed. We then chose random samples of all patients in 10% increments from 10% to 100%, simulating incomplete capture of all potential cases by the registry. For each sample we calculated the number of cases of the new implant predicted to end in revision. The assignments were repeated 2000 times using implants with revision rates of 1.5%, 2.0% and 3.0% per annum vs. 1.0% for all other implants of the same class.
RESULTS
The observed failure rate of the new implant averaged 2.0%, but varied from 0.7–3.8% over the 2000 trials, with 100% enrollment. With only 10% enrollment, the spread of failure rates increased to 0.0–7.8%, corresponding to a 152% increase in the variability of the observed revision rate. When enrollment was increased from 80% to 100%, the variability of the failure rate changed by only 9% from a range of 1.63% (1.23–2.86%) to 1.50% (1.30–2.80%) (90% CI). The reliability of detection of poorly performing implants improved dramatically with enrollment. With 70% enrollment, an implant with a 2.0% failure rate could be detected with 95% confidence, while a 3.0% implant became apparent with only 21% enrollment. Conversely, with even 100% enrollment it was not possible to identify implants with a 1.5% annual failure rate with 95% confidence.
CONCLUSIONS
-
If registries collect a truly representative sample of only 50–80% of the total patient population, there will be only a slight increase in the risk of overlooking an inferior outlier, including poorly-performing implants, compared to 100% patient capture.
-
Our results suggest that enrollment of every patient receiving a given treatment is not nearly as important as randomization of the sample subjected to analysis.
