Abstract
Aims
We aimed to assess the comparability of data in joint replacement registries and identify ways of improving the comparisons between registries and the overall monitoring of joint replacement surgery.
Materials and Methods
We conducted a review of registries that are full members of the International Society of Arthroplasty Registries with publicly available annual reports in English. Of the six registries which were included, we compared the reporting of: mean age, definitions for revision and re-operation, reasons for revision, the approach to analysing revisions, and patient-reported outcome measures (PROMs) for primary and revision total hip arthroplasty (THA) and hip resurfacing arthroplasty (HRA).
Results
Outcomes were infrequently reported for HRA compared with THA and all hip arthroplasties. Revisions were consistently defined, though re-operation was defined by one registry. Implant survival was most commonly reported as the cumulative incidence of revision using Kaplan-Meier survival analysis. Three registries reported patient reported outcome measures.
Conclusion
More consistency in the reporting of outcomes for specific types of procedures is needed to improve the interpretation of joint registry data and accurately monitor safety trends. As collecting additional details of surgical and patient-reported outcomes becomes increasingly important, the experience of established registries will be valuable in establishing consistency among registries while maintaining the quality of data.
Take home message: As the volume of joint replacements performed each year continues to increase, greater consistency in the reporting of surgical and patient-reported outcomes among joint replacement registries would improve the interpretation and comparability of these data to monitor outcomes accurately.
Cite this article: Bone Joint J 2016;98-B:442–51.
Clinical registries have gained recognition as sources of data, enabling the monitoring of large, diverse groups of patients to identify trends, fill gaps in evidence, and improve the quality and safety of healthcare delivery.1-4 With increased focus on accountability in health care,2 and advancements in treatment and technology, these data may be used by clinicians and policy makers responsible for the planning of healthcare and the allocation of resources.3
National joint replacement registries have become important for tracking the outcome of arthroplasties worldwide and providing feedback to surgeons and orthopaedic units.1,2,5,6 Increased monitoring of the outcome of arthroplasties can help to identify poorly performing implants or techniques early, leading to product recall and changes in treatments.1,2,6 Registries can therefore contribute to reducing complications and the costs of healthcare.7,8 For example, since its establishment in 1979, the Swedish Hip Arthroplasty Register has become an essential part of quality improvement programmes and health care decision-making in Sweden, through its efforts to monitor and improve the outcome of total hip arthroplasty (THA).8 Using registry data, it was estimated that 11 630 revision THAs were avoided in Sweden over ten years, corresponding to approximately $140 million in direct cost savings.8
Comparing outcomes across registries can help to identify best practices, strengthen the validity of results reported by individual registries, and identify conflicting trends.9 To be comparable, it is crucial for registries to collect high-quality data that are consistent with each other. The quality of data depends on the completeness of collection and the correct coding of operations within databases, as these factors affect the accuracy, precision, and risk of bias.10,11 In order to facilitate valid comparisons, the terminology and methods of analysis used by registries must be accurate and consistent. In addition, the statistical methods which are used to evaluate outcomes must be capable of handling longitudinal, incomplete data as well as varying follow-up times, sizes of populations, and patient characteristics.12,13 Furthermore, with surgical and technological advancements, it is also important for them to monitor and report outcomes by the type of implant, procedural design, and surgical approach.
Collaboration between international registries, including the International Society of Arthroplasty Registries (ISAR), International Consortium of Orthopaedic Registries (ICOR), European Arthroplasty Register (EAR), and Nordic Arthroplasty Register Association (NARA), has been established in recent years to promote the development and standardisation of data collection among registries. The ICOR and NARA conduct aggregate analyses that include data from several registries.14,15 By capturing a large number of devices from various registries worldwide, these organisations carry out comprehensive analyses that both combine and compare international data.
In efforts to improve the standardisation of data collection, elements of data have been categorised into different levels to differentiate core patient, procedural and implant-related elements (level I) from those that require more extensive collection, such as information on patient comorbidities and adverse events (level II), patient-reported outcome measures (PROMs) (level III), and diagnostic imaging (level IV).16 Although joint registries provide a platform through which these outcomes can be monitored on a large-scale and ongoing basis, collecting detailed data (i.e., levels II to IV) is logistically challenging and may adversely affect the completeness and quality of data. Nevertheless, collecting these data may be of value to registries. For instance, given that arthroplasties are primarily performed to reduce pain, improve mobility and health-related quality of life, PROMs have become recognised as essential measures of the appropriateness, timing, and success of arthroplasty surgery.16-19 In providing information which is complementary to implant survival, PROMs have also become useful for identifying underperforming implants20 and informing clinical and policy decisions.17,21 It has been suggested that improvements in the capture of PROMs among registries is needed.22
The purpose of this study was to assess the comparability of registry data by examining the definitions, the methods of analysis and the outcomes reported by established hip registries, and identify key elements that will improve longitudinal comparisons between registries and the overall monitoring of arthroplasty surgery. Although we have chosen to focus on hip registries, we believe our assessment will be representative of the reporting practices of other types of registries since many report on a wider scope of arthroplasty procedures.
Materials and Methods
Selection criteria
In order to ensure that well-established, high-quality registries were included, we searched for national joint registries that were full members of the ISAR. Full members of the ISAR include national registries receiving data from over 80% of hospitals, each of which reports over 90% of the procedures performed, with validation processes in place.13 Joint registries were identified by searching the ISAR directory and publications found on its website.12,23 Our inclusion criteria were: hip registries that were full members of the ISAR with current, publicly available annual reports in English. Of 41 ISAR members identified, six registries (15%) met our inclusion criteria: the Australian Orthopaedic Association National Joint Replacement Registry24 National Joint Registry of England, Wales and Northern Ireland,25 New Zealand Joint Registry,20 Norwegian Arthroplasty Register,26 Slovak Arthroplasty Register27 and Swedish Hip Arthroplasty Register.28 The Danish Hip Arthroplasty Register, Dutch Arthroplasty Register, and Finnish Arthroplasty Register were also identified as hip registries that were full members of the ISAR; however, these were excluded as we were unable to find full and current annual reports in English.
Data extraction
Two reviewers independently (SL and CL) extracted data from the most recent annual reports available using an electronic data extraction spreadsheet. Disagreements were resolved by consensus and, if needed, discussion with a third reviewer (KM). We extracted definitions and outcomes aligning with ISAR’s essential minimum dataset13 or where inconsistencies have previously been noted,2 including: whether the annual number of primary and revision procedures were recorded, reporting the mean age of patients, definitions for revision and re-operation, and the methods and metrics used to analyse revisions. We also determined whether the following reasons for revision commonly described in the literature2,29 were reported by each registry: dislocation, infection, fractures involving the bone and implant, aseptic loosening, and osteolysis. These adverse events were defined as they were presented in each annual report. We also examined whether the frequency and outcomes of THAs and hip resurfacing arthroplasties (HRAs) were reported separately. Due to differences in design that make HRA suitable for younger, more active patients, outcomes for these procedures have been shown to be different and require careful interpretation.29-33 In addition, one reviewer (SL) extracted information related to data collection procedures, the type of data collected by the registry (e.g. administrative, medical records, data collection forms), completeness of data, validation processes, procedure coding, as well as the evaluation of PROMs, including the type (i.e. generic or disease-specific), frequency, and reporting of these outcomes.
Results
Registry characteristics
Of the six included registries, the year of its establishment ranged from 1979 (Sweden) to 2003 (England and Wales (subsequently expanded to include Northern Ireland), Slovakia) (Table I). Each collected prospective data using collection forms (often paper-based) that were submitted to the registry by the orthopaedic units of hospitals. Data were validated against national patient registers, government administrative databases, or medical department records. Two registries (England, Wales and Northern Ireland, and New Zealand) also validated their data against data from the suppliers and distributors of prostheses. The National Joint Registry of England and Wales also used government data to supplement their data in the analysis of revisions due to poor compliance in their earlier years of operation, which potentially led to under-reporting of revisions.34 The coding of procedures varied across the five registries that provided this information in their annual report.
Table I
Summary of data collection, coverage, and coding of hip replacements among six full members of the International Society of Arthroplasty Registries
| Outcome | Australia (AOANJRR) | England & Wales (NJREW) | New Zealand (NZJR) | Norway (NAR) | Slovakia (SAR) | Sweden (SHAR) |
|---|---|---|---|---|---|---|
| Year of establishment | 1999 | 2003 | 1999 | 1987 | 2003 | 1979 |
| Year of report | 2013 | 2013 | 2013 | 2010 | 2011 | 2011 |
| Year of data | 2012 | 2012 | 2012 | 2009 | 2011 | 2010 |
| Data collection | Paper-based registry forms are completed in the theatre at time of surgery and submitted by hospitals to the registry monthly | Records are submitted to the registry by orthopaedic units | Data collection forms are completed in the operating theatre then submitted to the registry | Paper-based data collection forms | Data collection forms are submitted by surgical departments. 79% are submitted using an Implant Tracking System* | All public and private hospitals and most clinics report to the registry using a web application |
| Data covered | Complete national data since 2003. Initial validation resulted in 93% of registry records verified against health department data | 86.8% (90% in-year compliance) | 98% | 98% | ||
| Validation | Validated using state and territory health department data | Validated using data held in the HES and PEDW services and the number of implants sold | Validated using procedures submitted to the New Zealand Health Information Service and prosthesis suppliers | Validated using Ministry of Health of the Slovak Republic databases | Validated using the National Board of Health and Welfare Patient Register | |
| Procedure coding | ICD-10-AM codes | HES data are coded using OPCS-4 codes | Operations are presented with measuring codes (KVA-codes NF and QD) | |||
-
* Based on Global Trade Item Number barcodes and the Health Industry Business Communications Council system AOANJRR, Australian Orthopaedic Association National Joint Replacement Registry; NJREW, National Joint Registry of England and Wales; NZJR, New Zealand Joint Registry; NAR, Norwegian Arthroplasty Register; SAR, Slovak Arthroplasty Register; SHAR, Swedish Hip Arthroplasty Register; NJR, National Joint Registry of England and Wales; HES/PEDW, Hospital Episodes Statistics/Patient Episode Data for Wales; OPCS-4 codes, Office of Population; Censuses and Surveys classification of interventions and procedures, 4th revision; THA, total hip arthroplasty Blank cells indicate outcome was not reported or could not be found
Definitions for repeat surgery (revision vs re-operation)
Varying levels of detail were provided for the definition of revision. However, each registry’s definition involved the replacement of one or more components of a THA (Table II). The most common theme was that a revision involves the removal of a component, while two registries (Australia, New Zealand) also specify that a revision may involve the addition of a component. The Swedish Hip Arthroplasty Register was the only one to provide a definition for re-operation in addition to revision (Table II); a re-operation was defined as any surgical intervention related to a previously inserted component, whereas a revision was defined as the replacement or removal of all or part of a THA. Therefore, a revision is considered to be a type of re-operation by this registry.
Table II
Definition of revision and reoperation used by six joint replacement registries
| Procedure | Definition |
|---|---|
| Revision | |
| Australia (AOANJRR) | Re-operations of previous hip replacements where one or more of the prosthetic components are replaced, removed, or another component is added. Revisions include re-operations of primary partial, primary total or previous revision procedures |
| England & Wales (NJREW) | Operation performed to remove (and usually replace) one or more components of a total joint prosthesis for whatever reason |
| New Zealand (NZJR) | A new operation in a previously replaced hip joint during which one of the components are exchanged, removed, manipulated or added. It includes excision arthroplasty and amputation, but not soft tissue procedures. A two-stage procedure is registered as one revision |
| Norway (NAR) | Exchange or removal of total prosthesis or components |
| Slovakia (SAR) | Any operation replacing any component |
| Sweden (SHAR) | Entails that a previously hip-replaced patient undergoes a further operation where part or all of the implant is replaced or extracted |
| Re-operation | |
| Australia (AOANJRR) | |
| England & Wales (NJREW) | |
| New Zealand (NZJR) | |
| Norway (NAR) | |
| Slovakia (SAR) | |
| Sweden (SHAR) | Comprises all types of surgical intervention relating directly to an inserted hip implant. It may be that the implant is left untouched, or revised, when the entire implant or at least one of its components is changed or extracted |
-
AOANJRR, Australian Orthopaedic Association National Joint Replacement Registry; NJREW, National Joint Registry of England and Wales; NZJR, New Zealand Joint Registry; NAR, Norwegian Arthroplasty Register; SAR, Slovak Arthroplasty Register, SHAR, Swedish Hip Arthroplasty Register Blank cells indicate outcome was not reported or could not be found
Variability in procedure reporting
The total number of primary and revision THAs which were captured throughout the most recent year was reported by five registries (Table III). Five reported the number of primary THAs performed, while four reported the number of primary HRAs. The number of revision THAs and HRAs was reported by four and three registries, respectively. The number of revision HRAs was often reported as the cumulative number captured by the registry since its establishment, rather than for the most recent year.
Table III
Summary of the reporting of primary and revision procedures and mean age of patients captured by six joint replacement registries for the most recent year of data
| Outcome | Australia (AOANJRR) | England & Wales (NJREW) | New Zealand (NZJR) | Norway (NAR) | Slovakia (SAR) | Sweden (SHAR) | Total |
|---|---|---|---|---|---|---|---|
| Number of primary surgeries | |||||||
| All hip procedures | Y | Y | Y | Y | Y | 5/6 | |
| THA | Y | Y | Y | Y | Y | 5/6 | |
| HRA | Y | Y | Y | Y | 4/6 | ||
| Number of revision surgeries | |||||||
| All hip procedures | Y | Y | Y | Y | Y | 5/6 | |
| THA | Y | Y* | Y | Y | 4/6 | ||
| HRA† | Y | Y | Y | 3/6 | |||
| Mean age (yrs) | |||||||
| All hip procedures | Y | Y | Y‡ | Y | 4/6 | ||
| THA | Y | Y | Y | Y | Y | 5/6 | |
| HRA | Y | Y | Y | Y | 4/6 | ||
-
* Includes the cumulative number of revisions registered since implementation of the registry † Commonly reported as the cumulative number of procedures recorded since implementation of the registry ‡ Reported for all operations performed since implementation of the register AOANJRR, Australian Orthopaedic Association National Joint Replacement Registry; NJREW, National Joint Registry of England and Wales; NZJR, New Zealand Joint Registry; NAR, Norwegian Arthroplasty Register; SAR, Slovak Arthroplasty Register; SHAR, Swedish Hip Arthroplasty Register; NJR, National Joint Registry of England and Wales; THA, Total hip arthroplasty; HRA, Hip resurfacing arthroplasty; Y, reported Blank cells indicate outcome was not reported or could not be found
Each registry reported the mean age of patients, either for all arthroplasties (n = 4) or specific to THAs (n = 5) or HRAs (n = 4). Several registries also reported the number of patients within specific age categories that underwent arthroplasty surgery.
Methods for analysing revisions
The survival of the implant was reported using several metrics, including the revision burden, person-time incidence rates, and the cumulative incidence of revision (Table IV). Five registries reported the revision burden as the number of revisions divided by the total number of primary and revision operations performed during the most recent year. The New Zealand Registry reported the revision burden as the ratio of revision operations to primary operations (revisions:primaries) registered since the establishment of the registry. The Swedish Register reported the revision burden for all operations performed between 1979 and 2011, and 2002 and 2011, and also presented this information for each year since the registry’s establishment.
Table IV
Methods for analysing and reporting revisions among six joint replacement registries
| Outcome | Australia (AOANJRR) | England & Wales (NJREW) | New Zealand (NZJR) | Norway (NAR) | Slovakia (SAR) | Sweden (SHAR) | Total |
|---|---|---|---|---|---|---|---|
| Revision burden * | |||||||
| All hip procedures† | Y | Y‡ | Y | Y§ | 4/6 | ||
| THA | Y | Y | 2/6 | ||||
| HRA | Y | 1/6 | |||||
| Person-time incidence rate of revision | |||||||
| All hip procedures | Y¶ | 1/6 | |||||
| THA | Y¶ | Y** | 2/6 | ||||
| HRA | Y¶ | Y** | 2/6 | ||||
| Cumulative percent revised | |||||||
| All hip procedures | Y | Y | Y | Y | 4/6 | ||
| THA | Y | Y | Y | Y | Y | 5/6 | |
| HRA | Y | Y | Y | 3/6 | |||
-
* Calculated as the number of revisions performed divided by the total number of operations performed during the most recent year † Includes all hip procedures recorded by the registry (i.e., total and partial hip replacements and hip resurfacing) ‡ Calculated as the ratio of revision operations to primary operations (i.e., revisions: primaries) registered since the establishment of the registry §Calculated for operations performed from 1979 to 2011 and 2002 to 2011, and presented graphically for each year from 1979 to 2011 ¶ Calculated as the number of revisions per 1000 patient-years ** Calculated as the number of revisions per 100 component-years AOANJRR, Australian Orthopaedic Association National Joint Replacement Registry; NJREW, National Joint Registry of England and Wales; NZJR, New Zealand Joint Registry; NAR, Norwegian Arthroplasty Register; SAR, Slovak Arthroplasty Register; SHAR, Swedish Hip Arthroplasty Register; NJR, National Joint Registry of England and Wales; THA, total hip arthroplasty; HRA, Hip resurfacing arthroplasty; Y, reported Blank cells indicate outcome was not reported or could not be found
Person-time incidence rates were reported by two registries, calculated as either the number of revisions per 1000 patient-years (England, Wales and Northern Ireland) or per 100 component-years (New Zealand). Each registry used survival analysis to estimate the cumulative incidence of revision using the Kaplan-Meier method.35 These outcomes were described as the cumulative percent revised, cumulative percent not revised, cumulative risk of revision, or proportion free of revision. While five registries reported the cumulative incidence of revision for THAs, only three reported the cumulative incidence of revision for HRAs.
Reasons for revision
The reasons for revision were not mutually exclusive and were calculated as the percentage of all revisions performed for a specific reason during either the most recent year or since the establishment of the registry. Four registries reported reasons for revision for all types of primary THA, while only three reported the reasons for revision specific to THAs or HRAs (Table V). Of the reasons for revision which we examined, fracture of a component and osteolysis were the least commonly reported.
Table V
Summary of the reasons for revision reported by six joint replacement registries, either as a percentage of revisions performed in the most recent year (i.e., annual) or from implementation of the registry (i.e., cumulative)*
| Outcome | Australia (AOANJRR) | England & Wales (NJREW) | New Zealand (NZJR) | Norway (NAR) | Slovakia (SAR) | Sweden (SHAR) | Total |
|---|---|---|---|---|---|---|---|
| All hip procedures | |||||||
| Reporting period† | Annual | Annual | Annual | Cumulative | 4/6 | ||
| Dislocation | Y | Y | Y | Y | 4/6 | ||
| Infection | Y | Y | Y | Y | 4/6 | ||
| Bone fracture | Y | Y | Y‡ | Y | 4/6 | ||
| Fracture of a component | Y | Y | 2/6 | ||||
| Aseptic loosening | Y | Y§ | Y§ | Y | 4/6 | ||
| Osteolysis | Y | Y§ | Y§ | 3/6 | |||
| THA | |||||||
| Reporting period | Cumulative | Cumulative | Cumulative | 3/6 | |||
| Dislocation | Y | Y | Y | 3/6 | |||
| Infection | Y | Y | Y | 3/6 | |||
| Bone fracture | Y | Y | Y | 3/6 | |||
| Fracture of a component | Y | 1/6 | |||||
| Aseptic loosening | Y¶ | Y | Y§ | 3/6 | |||
| Osteolysis | Y¶ | Y | Y | 3/6 | |||
| HRA | |||||||
| Reporting period | Cumulative | Cumulative | Cumulative | 3/6 | |||
| Dislocation | Y | Y | Y | 3/6 | |||
| Infection | Y | Y | Y | 3/6 | |||
| Bone fracture | Y | Y | 2/6 | ||||
| Fracture of a component | Y | Y | 2/6 | ||||
| Aseptic loosening | Y¶ | Y | Y§ | 3/6 | |||
| Osteolysis | Y¶ | Y | 2/6 | ||||
-
*Reasons for revision are not mutually exclusive (one revision may have more than one adverse event) † This refers to the time horizon over which registries report outcomes. Annual refers to the most recent year of data. Cumulative refers to the total number of years since implementation of the registry ‡ Presented as ‘periprosthesis fracture’ § Presented separately for acetabular and femoral components ¶ Presented as ‘loosening/lysis’ AOANJRR, Australian Orthopaedic Association National Joint Replacement Registry; NJREW, National Joint Registry of England and Wales; NZJR, New Zealand Joint Registry; NAR, Norwegian Arthroplasty Register; SAR, Slovak Arthroplasty Register; SHAR, Swedish Hip Arthroplasty Register; THA, total hip arthroplasty; HRA, hip resurfacing arthroplasty; Y, reported Blank cells indicate outcome was not reported or could not be found
Patient reported outcome measures (PROMs)
Three registries (Sweden, England, Wales and Northern Ireland, and New Zealand) reported PROMs in their annual reports (Table VI). The Swedish Register began collecting generic and disease-specific PROMs through a nationwide system in 2010 using the European Quality of Life-5 Dimensions (EQ-5D)36 to measure health-related quality of life and visual analogue scales to measure pain and satisfaction. Pain and EQ-5D scores were reported pre-operatively and all three PROMs were reported at one and six years post-operatively. The National Joint Registry of England, Wales and Northern Ireland provided summary statistics for generic (EQ-5D Index, EQ-5D Health Scale) and disease-specific (Oxford hip score)37 PROMs collected pre-operatively and six months post-operatively, and reported trends in the differences over time. The New Zealand Registry sends a modified Oxford-12 questionnaire to a random selection of patients six months after the initial surgery and at five-year intervals thereafter. Based on the Oxford hip questionnaires, they ask questions regarding dislocation, infection, and other complications that do not require further surgery. The results are summarised as the proportion of patients scoring in the worst category for each question and are used to examine relationships between PROMs and two-year revision rates. Although the Norwegian Registry does not report PROMs in their annual report, it has implemented several studies related to PROMs.
Table VI
Generic and disease specific patient-reported outcome measures reported by six joint replacement registries
| Outcome | Australia (AOANJRR) | England & Wales (NJREW) | New Zealand (NZJR) | Norway (NAR) | Slovakia (SAR) | Sweden (SHAR) |
|---|---|---|---|---|---|---|
| PROM | ||||||
| Generic | EQ-5D Index, EQ-5D Health Scale | EQ-5D | ||||
| Disease-specific | Oxford hip Score | Oxford hip Score | Pain VAS, Satisfaction VAS | |||
| Data collection | ||||||
| Pre-operative questionnaire | Completed at time of surgery | Includes EQ-5D and pain VAS | ||||
| Post-operative questionnaire | Sent to patients at 6 months following surgery | Sent to a random selection of patients 6 months following surgery, then at 5 year intervals | Includes EQ-5D, pain VAS, and satisfaction VAS; sent to patients 1, 6 and 10 years following surgery | |||
-
AOANJRR, Australian Orthopaedic Association National Joint Replacement Registry; NJREW, National Joint Registry of England and Wales; NZJR, New Zealand Joint Registry; NAR, Norwegian Arthroplasty Register; SAR, Slovak Arthroplasty Register; PROM, patient-reported outcome measure; EQ-5D, European quality-of-life questionnaire-five dimensions; VAS, visual analogue scale Blank cells indicate outcome was not reported or could not be found
Discussion
Since the 1970s, joint registries have been established to monitor outcomes of arthroplasty surgery and have played a role in reducing post-operative complications, revision rates, and healthcare costs.7,8 Collectively comparing data across registries can be useful to identify strengths and weaknesses within individual registries, and thus improve their value. Several previous studies have compared the organisational structure or outcomes reported by joint registries,5,9,10,16,21,29,38,39 though difficulties in comparing or combining these data were frequently noted. Given the large number of registries that have recently been established, our goal was to provide a comprehensive summary of the current reporting practices of high-quality hip registries worldwide to highlight areas where there remain differences.
In particular, we assessed trends towards standardisation of measures among registries. We found that the six registries included in our study were consistent in reporting the annual number of primary and revision procedures as well as the revision burden for all hip replacement procedures. Outcomes specific to THAs and HRAs were reported less frequently. As outcomes for these procedures have been shown to vary, and as the use of HRA has recently increased specifically in young active males,29-33 it is important to assess outcomes based on the design of the procedure separately.
The definition for revision was similar among the registries included in our study. However, only one defined re-operation,28 and considered revisions to be a type of re-operation. Stricter definitions in the literature define a revision as the replacement of a component of a primary THA and re-operations as a subsequent operation in which a component is not replaced.29 The lack of standardised definitions to differentiate revisions from re-operations among registries currently limits the ability to code procedures accurately, and may reduce the ability to make valid comparisons between registries and the arthroplasty literature.29
The cumulative incidence of revision was the most common approach for reporting rates of revision in the registries included in our review. In illustrating how the probability of revision changes over time, the cumulative incidence is more useful than person-time incidence rates, which provide an overall estimate of the incidence of revisions per unit of time (i.e., per 1000 patient-years). Furthermore, person-time revision rates reported by national registries have been shown to be different to those in the literature, where the rates of revision in registries were generally lower.29 The cumulative incidence is also preferred over the revision burden, which does not consider the follow-up time in its calculation.
Estimating the cumulative incidence requires a set of statistical techniques called survival analysis to account for varying times of follow-up and incomplete (censored) data. Although Kaplan–Meier survival analysis is used consistently among the registries in our study, there are concerns regarding its use to estimate the cumulative incidence of revision, since it does not account for competing risks.40-42 Competing risks are events, such as death, that may preclude and alter the probability of revision.43 By assuming that patients who die are still at risk of revision, the Kaplan-Meier method overestimates the incidence of revision.24,34,40,44-46 While these estimates may be clinically relevant from an aetiological or patient perspective,44,45 methods that account for competing risks provide more accurate estimates of the proportion of patients revised over time and are therefore more useful for planning the delivery of health care and allocation of resources.41,45,46,47 As the appropriateness of each method depends on the perspective of the audience, it may be useful to report results obtained using both methods.
Even where standardised reports of incidence are used, different units of analysis (e.g., patients or components), definitions of outcomes, and completeness of data limit the comparability and generalisability of outcomes.29 Furthermore, several key determinants of the outcome after arthroplasty, including patient, surgeon, and hospital factors must be controlled for in order to make valid comparisons between different groups of patients. For instance, pre-operative pain and function,48-50 mental health status,49-51 and comorbidity,50 including other musculoskeletal symptoms,48,52 have been shown to influence pain and function after arthroplasty. In addition to enhancing the comparability of outcomes, improved collection of detailed pre- and post-operative information about patients in registries may lead to a better understanding of modifiable and non-modifiable predictors of outcomes. However, the increased burden of data collection and its negative consequences with respect to the completeness of data challenges the feasibility of collecting these additional elements.
Although revision is commonly used as an endpoint for measuring the success of arthroplasty surgery due to its ease of reporting, it may not provide the most objective or sensitive measure of clinical failure.16,19,53 For instance, factors such as referral for a revision, which depend on the individual surgeon and patient’s willingness to undergo further surgery, may influence the rates of revision.53 Although using PROMs to capture pain and functional outcomes may provide a better assessment of the success of arthroplasty surgery, there are challenges surrounding the collection and analysis of PROMs.17 First, given the variety of available PROMs, it is important that validated tools developed to meet psychometric tests are used, including disease-specific PROMs such as the Western Ontario and McMaster Universities Osteoarthritis Index54 to assess outcomes specifically related to the arthroplasty, and generic PROMs such as the EQ-5D36 to capture overall health outcomes.16,17 Additional complications regarding the analysis of PROMs include measuring and determining minimal clinically important differences between pre- and post-operative scores, accounting for potential confounding factors, the handling of non-normal data, ceiling effects of improvement scores, and completion bias. Thus, further consideration of the potential benefits, feasibility, and barriers to collecting PROMs for registries is required. The experience of registries which have previously, or currently, collect PROMs (i.e., Sweden, England, Wales and Northern Ireland, and New Zealand; as identified in our study and further described by Romero et al22) as well as five regional registries in the United States21 that recently implemented the collection of PROMs, will be valuable to exploring these issues further.
Difficulties in combining or comparing registry data have previously been noted.5,9,10,16,29,38,39 Marshall et al29 described challenges with comparing the outcomes after THA and HRA reported by registries and individual studies, and highlighted the importance of standardised definitions and outcome metrics. Migliore et al38 also reported difficulties in comparing outcomes of specific hip systems presented in the reports of registries. Robertsson et al10 emphasised the need for detailed, standardised data collection and analysis of knee arthroplasty registry data. Labek et al9 noted that greater co-operation between registries may improve the sharing of relevant information. The ISAR and ICOR have also noted that improved standardisation of terminology, methods of analysing survival and classification of implants is needed among registries.12-14,55 Nonetheless, there are trade-offs between standardised data collection and individualised reporting, and the different objectives of each registry may influence the data that are collected and reported.9 Unique and thoughtful reporting practices can be beneficial for drawing useful conclusions from data and in responding to individual national priorities. However, variation in methods of analysis and reporting must be clearly documented by registries to ensure that these differences are understood and appropriate conclusions drawn from registry data.
Our comprehensive review of several hip registries aimed to build upon these previously identified issues and highlight areas that remain problematic. Despite focusing specifically on hip registries, we expect the issues identified in our study also apply to other types of registries. We included full members of the ISAR to ensure that the registries included in our study represent best practice. Thus, the inconsistencies which we observed may be more extreme among developing registries with less rigorous data collection, analysis, and reporting practices. For example, we compared the results of this review with the Canadian Joint Replacement Registry (CJRR),56 which is an associate member of the ISAR. Associate ISAR members include developing national registries with < 80% of data coverage or no validation processes in place, as well as regional and multi-national registries.13 In general, we found the CJRR reported outcomes specific to types of procedure (i.e., THA and HRA) less frequently than the full ISAR members included in our study. For instance, the CJRR did not report the number of revisions, mean age of patients, reasons for revision, or revision burden specifically for THA and HRA. The CJRR also did not report implant survival using standardised person-time incidence rates or the cumulative incidence of revision.
Our review has limitations. We did not include non-English annual reports or studies published in the literature. Although studies published in peer-reviewed journals may be of higher quality in terms of study design (e.g., randomised controlled trials) and follow-up protocols, the credibility of registry data has increased recently.1-3 The ongoing, large-scale data collection facilitated by registries produces longitudinal, high-powered analyses that can be used to provide timely feedback to surgeons, orthopaedic units, and policy makers.1,2,5,6 However, we acknowledge that the information contained in annual reports available publicly represents only a subset of the data collected and analysed by each registry. Therefore, our review may not be representative of all information collected by each registry. Finally, we did not examine several areas of the collection, analysis and reporting of registry data that may confound or modify the outcomes which they report. For example, differences in population characteristics, the use, categorisation, and tracking of different brands of implant, stratified or joint analyses of groups of patients or brands of implant, and the handling of outliers when analysing implant survival by type, surgeon, or hospital may affect the comparability of the data.
Given the projected rise in the demand for THA, continued work is required among registries to further improve the standardisation of data collection, definitions, methods of analysis, and the reporting of outcomes by implant, procedural design, and patient characteristics to ensure that these data are comparable. Collaborations between international registries have improved these issues and have created a promising future for further collaboration. Consensus among registries as well as other stakeholders, including surgeons, patients, and healthcare decision makers, regarding what data should be collected and reported by registries will strengthen their ability to achieve their goals of accurately recording and monitoring outcomes across many groups of patients, regions, and countries.
1 Graves SE . The value of arthroplasty registry data. Acta Orthop2010;81:8–9. Google Scholar
2 Pivec R , JohnsonAJ, MearsSC, MontMA. Hip arthroplasty. Lancet2012;380:1768–1777. Google Scholar
3 Dreyer NA , GarnerS. Registries for robust evidence. JAMA2009;302:790–791. Google Scholar
4 Maloney WJ . The role of orthopaedic device registries in improving patient outcomes. J Bone Joint Surg [Am]2011;93-A:2241. Google Scholar
5 Macpherson GJ , BrenkelIJ, SmithR, HowieCR. Outlier analysis in orthopaedics: use of CUSUM: the Scottish Arthroplasty Project: shouldering the burden of improvement. J Bone Joint Surg [Am]2011;93(Suppl 3):81–88. Google Scholar
6 Robertsson O . Knee arthroplasty registers. J Bone Joint Surg [Br]2007;89-B:1–4. Google Scholar
7 Capozzi JD , RhodesR. Examining the ethical implications of an orthopaedic joint registry. J Bone Joint Surg [Am]2010;92-A:1330–1333. Google Scholar
8 Malchau H , HerbertsP, EislerT, GarellickG, SödermanP. The Swedish Total Hip Replacement Register. J Bone Joint Surg [Am]2002;84-A(Suppl 2):2–20. Google Scholar
9 Labek G , StoicaCI, BöhlerN. Comparison of the information in arthroplasty registers from different countries. J Bone Joint Surg [Br]2008;90-B:288–291. Google Scholar
10 Robertsson O , BizjajevaS, FenstadAM, et al.Knee arthroplasty in Denmark, Norway and Sweden. Acta Orthop2010;81:82–89. Google Scholar
11 Arts DGT , De KeizerNF, SchefferGJ. Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inform Assoc2002;9:600–611. Google Scholar
12 No authors listed. International Society of Arthroplasty Registers. International Society of Arthroplasty Registries (revised February 2007). http://www.isarhome.org/statements (date last accessed 8 January 2016). Google Scholar
13 No authors listed. International Society of Arthroplasty Registers. Bylaws (revised March 2013). http://www.isarhome.org/statements (date last accessed 8 January 2016). Google Scholar
14 Sedrakyan A , PaxtonEW, PhillipsC, et al.The International Consortium of Orthopaedic Registries: overview and summary. J Bone Joint Surg [Am]2011;93-A(Suppl 3):1–12. Google Scholar
15 Havelin LI , RobertssonO, FenstadAM, et al.A Scandinavian experience of register collaboration: the Nordic Arthroplasty Register Association (NARA). J Bone Joint Surg [Am]2011;93-A(Suppl 3):13–19. Google Scholar
16 Rolfson O , RothwellA, SedrakyanA, et al.Use of patient-reported outcomes in the context of different levels of data. J Bone Joint Surg [Am]2011;93-A(Suppl 3):66–71. Google Scholar
17 Hossain FS , KonanS, PatelS, Rodriguez-MerchanEC, HaddadFS. The assessment of outcome after total knee arthroplasty: are we there yet?Bone Joint J2015;97-B:3–9. Google Scholar
18 Rothwell AG , HooperGJ, HobbsA, FramptonCM. An analysis of the Oxford hip and knee scores and their relationship to early joint revision in the New Zealand Joint Registry. J Bone Joint Surg [Br]2010;92-B:413–418. Google Scholar
19 Kolling C , SimmenBR, LabekG, GoldhahnJ. Key factors for a successful National Arthroplasty Register. J Bone Joint Surg [Br]2007;89-B:1567–1573. Google Scholar
20 No authors listed. The New Zealand Joint Registry. Fourteen Year Report, January 1999 to December 2012. http://www.nzoa.org.nz/system/files/NJR%2014%20Year%20Report.pdf (date last accessed 8 January 2016). Google Scholar
21 Franklin PD , LewallenD, BozicK, et al.Implementation of patient-reported outcome measures in U.S. Total joint replacement registries: rationale, status, and plans. J Bone Joint Surg [Am]2014;96-A(Suppl 1):104–109. Google Scholar
22 Romero L , NieuwenhuijseM, CarrA, SedrakyanA. Review of clinical outcomes-based anchors of minimum clinically important differences in hip and knee registry-based reports and publications. J Bone Joint Surg [Am]2014;96-A(Suppl 1):98–103. Google Scholar
23 No authors listed. International Society of Arthroplasty Registers Directory. http://www.isarhome.org/directory (date last accessed 8 January 2016). Google Scholar
24 No authors listed. Australian Orthopaedic Association National Joint Replacement Registry Annual Report 2013. Adelaide: Australian Orthopaedic Association; 2013. https://aoanjrr.dmac.adelaide.edu.au/documents/10180/127202/Annual%20Report%202013?version=1.2& t=1385685288617 (date last accessed 8 January 2016). Google Scholar
25 No authors listed. National Joint Registry for England, Wales and Northern Ireland 10th Annual Report 2013. http://www.njrcentre.org.uk/njrcentre/Portals/0/Documents/England/Reports/10th_annual_report/NJR%2010th%20Annual%20Report%202013%20B.pdf (date last accessed 8 January 2016). Google Scholar
26 Furnes O , HavelinLI, EngesæterB, FenstadAMThe Norwegian Arthroplasty Register. Report June 2010. Bergen, Norway: 2009. http://nrlweb.ihelse.net/eng/Report_2010.pdf (date last accessed 8 January 2016). Google Scholar
27 Nečas L , KatinaS, UhlárováJ, ColtonCLSurvival analysis of total hip and knee replacement in Slovakia 2003–2011. Acta Chir Orthop Traumatol Cech 2013;80(Suppl):4-85. http://sar.mfn.sk/file/subory/Annual%20report%20-%202011.pdf (date last accessed 8 January 2016). Google Scholar
28 Garellick G , KarrholmJ, RogmarkC, RolfsonO, HerbertsPSwedish Hip Arthroplasty Register Annual Report 2011. Goteborg: Swedish Hip Arthroplasty Register; 2012. http://www.shpr.se/en/Publications/DocumentsReports.aspx (date last accessed 8 January 2016). Google Scholar
29 Marshall DA , PykermanK, WerleJ, et al.Hip resurfacing versus total hip arthroplasty: a systematic review comparing standardized outcomes. Clin Orthop Relat Res2014;472:2217–2230. Google Scholar
30 Simpson JM , VillarRN. Resurfacing registers concern. J Bone Joint Surg [Br]2010;92-B:1493–1497. Google Scholar
31 Smith TO , NicholsR, DonellST, HingCB. The clinical and radiological outcomes of hip resurfacing versus total hip arthroplasty: a meta-analysis and systematic review. Acta Orthop2010;81:684–695. Google Scholar
32 van der Weegen W , HoekstraHJ, SijbesmaT, et al.Survival of metal-on-metal hip resurfacing arthroplasty: a systematic review of the literature. J Bone Joint Surg [Br]2011;93-B:298–306. Google Scholar
33 MacKenzie JR , O’ConnorGJ, et al.Functional outcomes for 2 years comparing hip resurfacing and total hip arthroplasty. J Arthroplasty2012;27:750–757. Google Scholar
34 No authors listed. National Joint Registry for England and Wales 8th Annual Report, 2011. http://www.njrcentre.org.uk/njrcentre/Reports,PublicationsandMinutes/Annualreports/Archivedannualreports/tabid/87/Default.aspx (date last accessed 8 January 2016). Google Scholar
35 Kaplan EL , MeierP. Nonparametric Estimation from Incomplete Observations. J Am Stat Assoc1958;53:457–481. Google Scholar
36 No authors listed. The EuroQol Group. EuroQol-a new facility for the measurement of health-related quality of life Health Policy1990;16:199–208. Google Scholar
37 Dawson J , FitzpatrickR, CarrA, MurrayD. Questionnaire on the perceptions of patients about total hip replacement. J Bone Joint Surg [Br]1996;78:185–190. Google Scholar
38 Migliore A , PerriniMR, RomaniniE, et al.Comparison of the performance of hip implants with data from different arthroplasty registers. J Bone Joint Surg [Br]2009;91-B:1545–1549. Google Scholar
39 Paxton EW , FurnesO, NambaRS, et al.Comparison of the Norwegian knee arthroplasty register and a United States arthroplasty registry. J Bone Joint Surg [Am]2011;93-A(Suppl 3):20–30. Google Scholar
40 Lacny S , WilsonT, ClementF, et al.Kaplan-Meier survival analysis overestimates the risk of revision arthroplasty: a meta-analysis. Clin Orthop Relat Res2015;473:3431–3442. Google Scholar
41 Wongworawat MD , DobbsMB, GebhardtMC, et al.Editorial: estimating survivorship in the face of competing risks. Clin Orthop Relat Res2015;473:1173–1176. Google Scholar
42 Porcher R . CORR Insights: Kaplan-Meier Survival Analysis Overestimates the Risk of Revision Arthroplasty: A Meta-analysis. Clin Orthop Relat Res2015;473:3443–3445. Google Scholar
43 Gooley TA , LeisenringW, CrowleyJ, StorerBE. Estimation of failure probabilities in the presence of competing risks: new representations of old estimators. Stat Med1999;18:695–706. Google Scholar
44 Biau DJ , LatoucheA, PorcherR. Competing events influence estimated survival probability: when is Kaplan-Meier analysis appropriate?Clin Orthop Relat Res2007;462:229–233. Google Scholar
45 Fennema P , LubsenJ. Survival analysis in total joint replacement: an alternative method of accounting for the presence of competing risk. J Bone Joint Surg [Br]2010;92-B:701–706. Google Scholar
46 Ranstam J , KärrholmJ, PulkkinenP, et al.Statistical analysis of arthroplasty data. II. Guidelines. Acta Orthop2011;82:258–267. Google Scholar
47 Lau B , ColeSR, GangeSJ. Competing risk regression models for epidemiologic data. Am J Epidemiol2009;170:244–256. Google Scholar
48 Hawker GA , BadleyEM, BorkhoffCM, et al.Which patients are most likely to benefit from total joint arthroplasty?Arthritis Rheum2013;65:1243–1252. Google Scholar
49 Judge A , ArdenNK, CooperC, et al.Predictors of outcomes of total knee replacement surgery. Rheumatology (Oxford)2012;51:1804–1813. Google Scholar
50 Lingard EA , KatzJN, WrightEA, SledgeCB. ; Kinemax Outcomes Group. Predicting the outcome of total knee arthroplasty. J Bone Joint Surg [Am]2004;86-A:2179–2186. Google Scholar
51 Singh JA , LewallenDG. Medical and psychological comorbidity predicts poor pain outcomes after total knee arthroplasty. Rheumatology (Oxford)2013;52:916–923. Google Scholar
52 Perruccio AV , PowerJD, EvansHM, et al.Multiple joint involvement in total knee replacement for osteoarthritis: effects on patient-reported outcomes. Arthritis Care Res (Hoboken)2012;64:838–846. Google Scholar
53 Goodfellow JW , O’ConnorJJ, MurrayDW. A critique of revision rate as an outcome measure: re-interpretation of knee joint registry data. J Bone Joint Surg [Br]2010;92-B:1628–1631. Google Scholar
54 Bellamy N , BuchananWW, GoldsmithCH, CampbellJ, StittLW. Validation study of WOMAC: a health status instrument for measuring clinically important patient relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee. J Rheumatol1988;15:1833–1840. Google Scholar
55 Robertsson O , MendenhallS, PaxtonEW, InacioMC, GravesS. Challenges in prosthesis classification. J Bone Joint Surg [Am]2011;93-A(Suppl 3):72–75. Google Scholar
56 No authors listed. Hip and Knee Replacements in Canada: Canadian Joint Replacement Registry 2013 Annual Report. Toronto, ON: Canadian Institute for Health Information; 2013. https://secure.cihi.ca/free_products/CJRR_2013_Annual_Report_EN.pdf (date last accessed 8 January 2016). Google Scholar
Author contributions:
S. Lacny: Study conception and design, acquisition, analysis and interpretation of the data, drafting and revising the manuscript, final approval of the version to be published.
E. Bohm: Study conception, interpretation of the data, revising the manuscript, final approval of the version to be published.
G. Hawker: Interpretation of the data, revising the manuscript, final approval of the version to be published.
J. Powell: Interpretation of the data, revising the manuscript, final approval of the version to be published.
D. A. Marshall: Study conception and design, analysis and interpretation of the data, revising the manuscript, final approval of the version to be published.
S. Lacny was supported by the Canadian Institutes of Health Research Master’s Award, Alberta Innovates – Health Solutions Graduate Studentship Award, and the Alberta Innovates – Health Solutions Osteoarthritis Team Trainee Studentship Award. D. A. Marshall is a Canada Research Chair in Health Systems and Services Research and Arthur JE. Child Chair in Rheumatology. These funders had no involvement in the study design, collection, analysis, or interpretation of the data.
No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article.
E. Bohm is the chair of the advisory committee of the Canadian Joint Replacement Registry.
We thank K. MacDonald and C. Lawrence for their assistance with data extraction.
This article was primary edited by J. Scott and first proof edited by G. Scott.
