Validation of primary metal-on-metal hip arthroplasties on the National Joint Registry for England, Wales and Northern Ireland using data from the London Implant Retrieval Centre

Patients and Methods

The aims of this study were to demonstrate a methodology for the retrieval and linkage of data from the registry and to perform independent validation and quality assessment of data relating to the records of primary procedures in the NJR. We performed a cross-sectional survey of the databases of the NJR and the London Implant RC on 5 November 2013. The study had ethical approval.

A total of 67 045 primary metal-on-metal hip procedures associated with a ten-digit NHS number⁷ were identified from the NJR database and 929 explanted pairs of components from the RC. Procedures performed outside the geographic area or temporal remit of the NJR or missing key fields to confirm linkage were excluded. NJR recording of primary hip arthroplasties for England and Wales commenced on 1 April 2003 and Northern Ireland joined the registry on 1 February 2013.¹

Linkage of NJR and RC records was performed using the NHS number and confirmed with a manual check of the patient’s name. Discrepancies were investigated using NHS Spine, which is a collection of national applications that support exchange of information across the NHS.⁸ Errors on the RC database were corrected at this stage using NHS Spine and/or direct liaison with the referring hospital or patient.

Linkage of the procedure was performed for patients using an exact match for the date of the operation and confirmed by manual checks of the manufacturer, type of implant, side, the size and reference number of the components and the name fields of the hospital. An exact match on four fields was required for linkage. This process was repeated to increase the number of linked records, by relaxing date constraints to +/- 12 months and including records with missing date field entries.

Linkage of the fields was achieved by identifying those which were to be paired between the two databases. RC fields were recoded to correspond with NJR codes where required (Table I).

Results

Of the 929 RC records, 52 were excluded because the primary procedure was performed before 1 April 2003 (England and Wales) or 1 February 2013 (Northern Ireland). Of the remaining 877 records, 518 (59.1%) had an NHS number that was found on the NJR database. There were 95 records that were subsequently excluded from the entire dataset due to missing primary operation date on the RC database, when linkage could not be performed. The result was that 476 of 782 (60.9%) records on the RC were successfully linked to the NJR primary database. Therefore, 306 (39.1%) records from 136 surgical units did not meet the criteria for linkage (Fig. 1). The rate of linkage improved with time.

Fig. 1

Study flowchart

Stage 1: Detailed data validation

This is provided in Tables II (NJR) and III (RC). There were no out of range or invalid values for any of the variables analysed. There were no duplicated records. The NJR had nearly perfect (> 99.9%) rates of completion of data for all fields. The lower rate for the size of the acetabular component presented in Table II was due to the use of a shell rather than a monobloc component, with only one true incomplete field entry.

Table II

Detailed data validation of National Joint Registry (NJR) primary database

Field	Valid	OOR	Missing	Missing rate (%)	Mean/count	sd	Min	Max
Age at primary (yrs)	67045	0	0	0.00	58.88	11.12	12	105
Gender	67040	0	5	0.00	Male: 40661	-	-	-
					Female: 26379
Primary operation date	67045	0	0	0.00	-	-	-	-
Surgical unit description	67045	0	0	0.00	417 units	-	-	-
Acetabular component brand	67044	0	1	0.00	51 brands	-	-	-
Acetabular component ODEP rating	67044	0	1	0.00	10A: 22240	-	-	-
					7A: 13354
					7B: 144
					5A: 7497
					5B: 2
					3A: 9342
					Discontinued: 6346
					Pre-entry: 7372
					Unclassified: 747
Component type	67044	0	1	0.00	Resurfacing: 33996	-	-	-
					Modular: 33048
Side	67045	0	0	0.00	Left: 31584	-	-	-
					Right: 35461
Head size (mm)	67044	0	1	0.00	44.90	6.98	22.25	60
Acetabular component size (Inner diameter) (mm)	51309	0	15 736 (All shell only)	23.5	47.95	4.61	28.00	64
Head cat number	67044	0	1	0.00	-	-	-	-
Head batch number	66958	0	87	0.1	-	-	-	-
Acetabular component cat number	67044	0	1	0.00	-	-	-	-
Acetabular component batch number	66952	0	93	0.1	-	-	-	-
Outcome code	67044	0	1	0.00	Revised: 5615	-	-	-
					Unrevised: 58257
					Death: 3172

1. Missing rate %, number missing / total number of records x 100; Max, maximum; Min, minimum; OOR, out of range; sd, standard deviation; Valid, total number of records (n = 67 045); number OOR, number missing Descriptive statistics for 67 045 records of metal-on-metal primary procedures recorded on the National Joint Registry from 1 April 2003 to 1 November 2013. Records without NHS numbers were excluded as they were not eligible for linkage

Table III

Detailed data validation of linked Retrieval Centre (RC) records.

Field	Valid	OOR	Missing	Missing rate (%)	Mean/count	sd	Min	Max
Age at primary (yrs)	418	0	58	12.2	56.29	10.32	25	107
Gender	476	0	0	0.00	Male: 209
					Female: 267
Primary operation date	418	0	58	12.2	-
Surgical unit description	445	0	31	6.5	170 units
Acetabular component brand	476	0	0	0.00	15 brands
Acetabular component ODEP rating	476	0	0	0.00	10A: 167
					7A: 27
					7B: 0
					5A: 77
					5B: 0
					3A: 47
					Discontinued: 115
					Pre-entry: 42
					Unclassified: 1
Component type	476	0	0	0.00	Resurfacing: 292
					Modular: 184
Side	466	0	10	2.1	Left: 226
					Right: 240
Head size (mm)	459	0	17	3.6	46.34	4.59	35	58
Acetabular component size (outer diameter) (mm)	452	0	24	5.0	53.33	3.91	44	66
Head cat number	241	0	Total: 235	49.4	-
			No head: 5
			Not visible: 230
Head batch number	245	0	Total: 231	48.5	-
			No head: 5
			Not visible: 226
Acetabular component cat number	295	0	Total: 181	38.0	-
			No cup: 24
			Not visible: 157
Acetabular component batch number	373	0	Total: 103	21.6	-
			No cup: 24
			Not visible: 79
Outcome code	476	0	0	0.00	Revised: 476
					Unrevised: 0
					Death: 0

1. Missing rate %, number missing / total number of records x 100; Max, maximum; Min, minimum; OOR, out of range; sd, standard deviation; ODEP, Orthopaedic Data Evaluation Panel; Valid, total number of linked records (n = 476); number OOR, number missing Descriptive statistics for 476 records linked between National Joint Registry and RC. Linkage was performed on a patient-level using NHS number and then on a procedure-level as indicated in the methods

The RC had > 85% data completion for 11 of 15 fields. The four component reference number fields had the poorest completion of data. Missing data were mainly due to reference numbers not being visible on the retrieved component. The remaining missing data were due to only a single component from an acetabular component: femoral head pairing being sent to the RC. For missing category numbers, 157 of 181 (86.7%) references for the acetabular component and 230/235 (97.9%) references for the femoral head were due to reference numbers not being visible. Reference numbers on femoral heads were less well completed than for acetabular components, commonly due to bone on the retrieved femoral head obscuring the reference (Fig. 2). Other reasons included reference numbers scratched off (in vivo or during explantation) or never etched on the component during manufacture.

Figs. 2a - 2b

Photographic examples of retrieved components with easily readable and unreadable component reference numbers showing a) femoral head components and b) acetabular components.

Discussion

This study has provided the first large-scale linkage of registry and retrieval data in orthopaedic surgery. It represents an independent assessment of the quality of data in the NJR in regards to one aspect of the registry, namely primary procedures undertaken on metal-on-metal hip arthroplasties. We found that NJR data for the primary procedure had nearly perfect completion. However, we were unable to link more than one third of primary procedures recorded on the RC (306/782 procedures (39.1%)) and identified a high error rate for outcome coding on the NJR (79/476 erroneous records (16.6%)). These two findings may represent an underestimation of the rate of revision for many types of metal-on-metal components on the NJR. This is an important finding due to the influence of published NJR data on our understanding of the surgical, implant design and patient factors that affect performance in joint replacement surgery. These errors will be particularly important if the NJR expands its reporting of surgeon-level data and may be a major factor in the interpretation of the activity of a given surgeon.

The absence of a Unique Device Identifier (UDI)¹² meant that we required matches on many criteria in order to create a link between a primary procedure and an explanted component. The initial ability to match on a patient-level using the unique NHS number allowed us to restrict the number of possible linkages for a given patient. This stage also excluded 359 patients from the possibility of any linkage. These patients either did not have their primary procedure reported to the NJR or had it reported without an NHS number. This may be consistent with the poor compliance and ‘linkability’ reported by the NJR in its early years. For example, compliance was only 64.7% and linkability 20% in 2003 (Fig. 3).^1,2 However, the fewest number of records were entered during this period.

Fig. 3

Histogram showing that 476 out of 782 eligible procedures on the Retrieval Centre database were linked to a primary procedure recorded with the National Joint Registry (NJR). Linkage rates were poor for the early years of the NJR and improved considerably from 2005. This is consistent with poor compliance and linkability reported in early NJR reports.

Strict criteria to accept linkage were essential to avoid making an incorrect link. The date of the primary procedure was a key field for linkage, but was often poorly or inaccurately completed by surgeons submitting to the RC. This may be because revision surgeons were often not responsible for and/or from units remote to the primary procedure. We did not perform linkage where there was uncertainty. We are extending the verification of the source of the initial operation as part of our retrieval analyses in order to improve the recording of the date of the primary procedure.

The NJR requires linkage of a primary procedure to a revision procedure for an outcome code to be changed from ‘unrevised’. This is reliant on compliance from the unit where the revision is undertaken, patient consent and linkability. Linkability is defined as the number of records associated with an NHS number divided by the total number of records on the NJR. The NJR 11th annual report quotes compliance of 99%, patient consent of 91.8% and linkability of 95.1% for 2013/14.¹

We identified one record incorrectly classified as ‘dead’, rather than ‘revised’. This is an important case to highlight given the forthcoming publication of surgeon-specific risk-adjusted mortality. We doubt that this is an isolated case, and even small numbers of errors may have a marked impact on the rates of mortality for low-volume arthroplasty surgeons. The ability of NJR data to account for confounders adequately is disputed.¹³ Surgeons are given the opportunity to review their data using a feedback system available through the password-protected section of the NJR website. Our findings suggest that it is advisable for surgeons to validate their own records and to inform the NJR of any discrepancies.

The number of revision procedures referred to the RC and not the NJR is difficult to quantify. It might be expected that where a surgeon has made a voluntary referral to the RC, he or she would also have fulfilled their compulsory obligation to refer to the NJR. However, whilst the RC deals almost exclusively directly with the referring surgeon, the NJR is typically provided with data through a system of data entry clerks. This chain may be responsible for some of the unlinked cases.

The highest error rate for any field was in the recording of the size of the femoral head (21.4%). Examination of these records found that 83/98 (84.7%) discrepancies were due to rounding down of odd-numbered ASR head sizes by 1 mm to the nearest even number. The two studies where NJR data were used to measure the effect of the size of the femoral head on the failure rate of metal-on-metal hip arthroplasties^14,15 are unaffected by this because they used even-numbered groupings.

The remaining discrepancies for this field and errors for the category number fields are of greater clinical significance because they reflect a different component being recorded on the NJR to that which was retrieved Table VI. However, more precise quantification is required. This is an extensive process that will require analysis of individual patient records, investigation of the processes for the collection of NJR data and examination of NJR implant catalogues.

We acknowledge that failure to identify incorrect linkage is not impossible with the design of this study. However, since we used strict criteria and performed manual checks, we feel that this is unlikely. The requirement for a perfect match to link records would eliminate the problem of incorrect linkage, but would also mask errors and so was not performed.

Errors in the batch number fields were due to fictitious data being entered into free-text fields on the NJR. These are important discrepancies, because they may prevent the possibility to trace patients who have received an implant from a defective batch. We are not aware of any manufacturer that publishes guidance on how it formats batch numbers. We recommend that this information should be available in the public domain and supplied to both the NJR and RCs.

Errors in the date of birth field were exclusive to the RC and found to be easily corrected using NHS spine. This process has since been incorporated into the source verification protocols undertaken by the RC. Discrepancies in the recording of the date of the primary procedure are also likely to lie with the RC, given that referrals are from the revision surgeon, as discussed above. The NJR faces the same difficulties with unlinked procedures on their revision database. Our early analyses suggest that the RC performs similarly to the NJR revision database for this field.

This study has presented novel methodology for the validation of a hip arthroplasty register using retrieved implants. The methods can be generalised to other regions where a RC has been established and unique patient identifiers are used. The study provides particularly robust validation for component fields. The careful cataloguing of retrieved components renders them available for scrutiny indefinitely.

We acknowledge that this is a cross-sectional survey and provides observational data. However, reports from arthroplasty registers are a major influence on the interpretation of the performance of arthroplasties¹⁶ and it is essential that any data upon which reports are based are entirely accurate. We recognise the difference in scale between the RC and the NJR. Linked cases in this study represent approximately 1% of all primary metal-on-metal hip procedures recorded in the NJR.

The RC is independent of the NJR. Linkage to the NJR was not contemplated when the RC began collecting implants and the rationale for collecting implants is different to the NJR. The primary aim of the RC is to determine the mechanism, rather than the rate, of failure. In addition, it is a different type of organisation, being a surgeon-led research collaboration within an NHS hospital – university partnership, rather than a government initiative aimed at governance and funded through industry.

Registry-retrieval linkage provided a novel means for validation of one aspect of an arthroplasty register. We found nearly perfect completion for NJR records of primary metal-on-metal hip arthroplasties. However, the NJR is likely to have underestimated rates of revision for metal-on-metal hips. A record of the primary procedure was not identifiable on the NJR in 39.1% of retrieved components. The NJR incorrectly coded the outcome following primary surgery in 16.6% of linked cases. These errors are relevant given the increasing scope for the application of NJR data. We recommend a system for continuous independent evaluation of the quality and validity of NJR data. Surgeons should ensure that their information is accurately recorded.