Abstract
Aims
This pilot study tested the performance of a rapid assay for diagnosing prosthetic joint infection (PJI), which measures synovial fluid calprotectin from total hip and knee revision patients.
Methods
A convenience series of 69 synovial fluid samples from revision patients at the Norfolk and Norwich University Hospital were collected intraoperatively (52 hips, 17 knees) and frozen. Synovial fluid calprotectin was measured retrospectively using a new commercially available lateral flow assay for PJI diagnosis (Lyfstone AS) and compared to International Consensus Meeting (ICM) 2018 criteria and clinical case review (ICM-CR) gold standards.
Results
According to ICM, 24 patients were defined as PJI positive and the remaining 45 were negative. The overall accuracy of the lateral flow test compared to ICM was 75.36% (52/69, 95% CI 63.51% to 84.95%), sensitivity and specificity were 75.00% (18/24, 95% CI 53.29% to 90.23%) and 75.56% (34/45, 95% CI 60.46% to 87.12%), respectively, positive predictive value (PPV) was 62.07% (18/29, 95% CI 48.23% to 74.19%) and negative predictive value (NPV) was 85.00% (34/40, 95% CI 73.54% to 92.04%), and area under the receiver operating characteristic (ROC) curve (AUC) was 0.78 (95% CI 0.66 to 0.87). Patient data from discordant cases were reviewed by the clinical team to develop the ICM-CR gold standard. The lateral flow test performance improved significantly when compared to ICM-CR, with accuracy increasing to 82.61% (57/69, 95% CI 71.59% to 90.68%), sensitivity increasing to 94.74% (18/19, 95% CI 73.97% to 99.87%), NPV increasing to 97.50% (39/40, 95% CI 85.20% to 99.62%), and AUC increasing to 0.91 (95% CI 0.81 to 0.96). Test performance was better in knees (100.00% accurate (17/17, 95% CI 80.49% to 100.00%)) compared to hips (76.92% accurate (40/52, 95% CI 63.16% to 87.47%)).
Conclusion
This study demonstrates that the calprotectin lateral flow assay could be an effective diagnostic test for PJI, however additional prospective studies testing fresh samples are required.
Cite this article: Bone Joint Res. 2020;9(5):202–210.
Article focus
-
Microbiological diagnosis of prosthetic joint infection (PJI) relies on culture techniques that are slow and insensitive. Inflammation biomarkers such as calprotectin have the potential to rapidly diagnose infection. This pilot study tests the performance of a calprotectin lateral flow assay for the diagnosis of PJI using synovial fluid samples.
Key messages
-
The lateral flow test has a high negative predictive value (NPV) compared to International Consensus Meeting (ICM) criteria, useful for ruling out infection. The test is highly accurate for diagnosing PJI when compared to a clinical review-based gold standard (ICM-CR).
-
The test may be more accurate for diagnosing PJI in knees than in hips.
-
Metallosis and severe osteolysis may be contraindications for use of the test.
Strengths and limitations
-
All eligible samples (sufficient synovial fluid volume and sufficient data to make ICM diagnosis) were tested, avoiding any selection bias, and rapid test operators were blind to gold standard results.
-
Samples were frozen and tested retrospectively and the number of knee samples tested was relatively low.
Introduction
Each year in the UK, approximately 160,000 primary hip and knee arthroplasties are performed plus an additional 14,000 revision operations. Prosthetic joint infection (PJI) is responsible for 14% of these revisions.1
Diagnosis of PJI remains a challenge and guidelines vary between countries. Low-grade PJIs are particularly difficult to diagnose. They are commonly caused by bacteria which do not have a clear pathogenic role and often contaminate tissue samples, making culture results difficult to interpret.2 These organisms are also less likely to trigger an increase in inflammatory markers.3 The inability to differentiate between low-grade infections and aseptic loosening leads to patients undergoing numerous investigations and unnecessary two-stage revisions. This treatment option comes with a higher cost both to the healthcare system and accompanying patient morbidity, along with an associated higher complication rate than one-stage revisions.4 For this group of patients especially there is a need for a diagnostic test that can reliably exclude PJI.
The Musculoskeletal Infection Society (MSIS) and International Consensus Meetings (ICMs) have published lists of criteria designed to standardize the diagnosis of PJI.5–7 The criteria require a diagnostic workup combining preoperative and intraoperative findings with selected inflammatory markers and culture results. MSIS and ICM criteria are useful but fallible, due to the heavy weighting placed on microbiological culture results, which are known to have suboptimal specificity and sensitivity.8,9
In response to the rapid emergence of new biomarkers for PJI,10–12 the ICM criteria were revised to include serum D-dimer, synovial alpha-defensins, and synovial leucocyte esterase in 2018.7 These tests can be expensive and their application in clinical practice is currently limited. An alternative biomarker is synovial fluid calprotectin. Calprotectin is a protein complex released during inflammation that makes up 60% of all soluble proteins in neutrophils.13 Neutrophils are recruited to sites of inflammation and infection response; therefore, high levels of neutrophil biomarkers are expected to be seen in infected patient samples.14
Calprotectin is routinely used to screen for inflammatory bowel disease and has been shown to detect relapse rheumatoid arthritis.15,16 In a recent study, a stool calprotectin test was used off label for PJI diagnosis, demonstrating a NPV of 94.4%.17 This demonstrates the utility of calprotectin for PJI diagnosis and the need for a validated lateral flow test.
Lyfstone AS (Tromsø, Norway) have recently developed a lateral flow calprotectin test for the diagnosis of PJI which has passed the European in vitro diagnostic (IVD) regulatory approval process (98/79/EC). We conducted the first clinical evaluation of this new technology in a retrospective study on samples from suspected PJI or aseptic loosening cases to assess the sensitivity, specificity, NPV, and positive predictive value (PPV).
Methods
A total of 73 synovial fluid samples that had been stored in the Biorepository at the University of East Anglia (Norwich, UK) between February 2016 and January 2019 were retrospectively tested for calprotectin levels with the Lyfstone calprotectin lateral flow test (Lyfstone AS). Samples were intraoperative specimens from a consecutive series of patients during revision total hip and total knee arthroplasty surgery performed at the Norfolk and Norwich University Hospital (NNUH; Norwich, UK). Samples were only placed in the Biorepository if patient consent was obtained at the time of surgery and sufficient sample remained following routine clinical evaluation.
Samples were only included for testing with the Lyfstone test if there was a sufficient volume (≥ 100 µl) and results for microbiological culture, frozen-section histology, and serum CRP were available. Subsequently, four samples were removed from the analysis due to incomplete/erroneous patient data (two had no preoperative notes and two were from a patient who had two separate surgeries, although the samples had the same date). A final total of 69 samples were included in the dataset (37 male, 32 female; mean age 74.3 years (45 to 89); mean body mass index (BMI) 29.8 kg/m2 (19.7 to 42.4)); 52 were taken during hip revision surgery and 17 from knee revisions.
Cases were classified as either infected or aseptic based on the new definition of prosthetic hip and knee infection from the 2018 ICM, hereafter referred to as ‘ICM’ criteria (Table I). A case was deemed as infected if the patient presented either with a major criterion (two or more cultures of the same organism or the presence of a sinus tract) or met three minor criteria (elevated serum CRP levels, purulence, one positive microbiological culture, and/or positive histological analysis of periprosthetic tissue for inflammation). Other minor criteria from the 2018 ICM criteria were not considered as many of those tests are not used as standard in the UK or routinely at NNUH. Clinical information was not made available to performers of the index test before testing.
Table I.
Scoring-based definition for prosthetic joint infection using available tests from the 2018 International Consensus Meeting criteria
| Criteria | Score | Decision |
|---|---|---|
| Major | Infected | |
| Two positive cultures of the same organism | N/A | |
| Sinus tract with evidence of communication to the joint or visualization of the prosthesis | N/A | |
| Minor | ≥ 6 infected; < 6 aseptic | |
| Elevated serum CRP (≥ 10 mg/l) or D-Dimer (D-Dimer unavailable) | 2 | |
| Positive histology | 3 | |
| Positive purulence | 3 | |
| Single positive culture | 2 | |
| Elevated serum ESR (unavailable) | 1 | |
| Elevated synovial WBC count or LE (unavailable) | 3 | |
| Positive alpha-defensin (unavailable) | 3 | |
| Elevated synovial PMNs (%) (unavailable) | 2 | |
| Elevated synovial CRP (unavailable) | 1 |
-
LE, leucocyte esterase; N/A, not applicable; PMN, polymorphonuclear neutrophil; WBC, white blood cell.
Cases where the calprotectin result was discordant with the ICM diagnosis of infection were further investigated by one of the senior authors (IM – Consultant Orthopaedic Surgeon) and team (MS, AD). The team were blinded to the calprotectin result and were asked to review the patient medical records, surgical notes, and radiographs. As this was a retrospective examination by the team, contemporaneous records from the treating surgeon (IM, MS, AD) were reviewed to ascertain the clinical decisions made at the time, for example a patient who had met the criteria for infection with ICM by two positive microbiology samples may not have been treated for infection as it was thought that those bacteria were contaminants (following multidisciplinary team discussion with the microbiologist at the time). The longer-term clinical outcome of the patient was then reviewed using one-year follow-up notes to ascertain if the clinical decision had been justified. We classified these results as ICM with Clinical Review - ICM-CR (aseptic loosening was graded according to the Paprosky classification).18
The Lyfstone calprotectin test was carried out according to the manufacturer’s kit instructions (Lyfstone AS). Synovial fluid samples (20 µl) were diluted in a premixed dilution buffer (2 ml – 101 × dilution) and added to the test cassette (80 µl). Gold-conjugated antibody complexes then bound calprotectin and travelled along the membrane within the cassette and further bound to immobilized calprotectin-specific antibodies to form a visible test line. Any remaining gold-conjugated antibody not bound to calprotectin was immobilized on a control line. After 15 minutes’ incubation at room temperature the cassette was photoimaged, and the calprotectin level was calculated by the Lyfstone smartphone application (Lyfstone AS) (Figure 1). The colour intensity of the test line was proportional to the concentration of calprotectin in the sample. A calprotectin result of ≤ 14 mg/l was considered negative according to the cutoff between low and moderate risk of infection by the Lyfstone smartphone application at the time of testing. Moderate (14 mg/l to 50 mg/l) and severe (> 50 mg/l) risk of infection categories were grouped together as positive.19
Fig. 1
Lyfstone application report (right) and Lyfstone calprotectin test (left) (Lyfstone AS, Tromsø, Norway).
Microbiological culture was performed on all tissue and fluid specimens (recommended that three to five tissue/fluid samples sent for testing) for 48 hours on blood agar and chocolate agar, five days on fastidious anaerobe agar, and five days in cooked meat broth before 48-hour subculture on fastidious anaerobe agar, chocolate agar, sabouraud agar, and in fastidious antimicrobial neutralization bottles according to the UK Standards for Microbiology Investigations B 44.20 Histology was performed by frozen section microscopy according to UKAS ISO 8405.21 Serum CRP was performed according to UKAS ISO 10295.22
Statistical analysis
All statistical calculations were performed using MedCalc Statistical Software version 19.1 (MedCalc Software, Ostend, Belgium). Statistical significance was set at p < 0.05.
Results
Lyfstone test performance compared to ICM
The overall sensitivity and specificity of the calprotectin test for the diagnosis of PJI compared to ICM criteria were 75.00% (18/24, 95% CI 53.29% to 90.23%) and 75.56% (34/45, 95% CI 60.46% to 87.12%), respectively. The PPV was 62.07% (18/29, 95% CI 48.23% to 74.19%), the NPV was 85.00% (34/40, 95% CI 73.54% to 92.04%), and the accuracy was 75.36% (52/69, 95% CI 63.51% to 84.95%) (Tables II and III). The area under the receiver operating characteristic (ROC) curve (AUC) was 0.78 (95% CI 0.66 to 0.87) (Figure 2).
Table II.
International Consensus Meeting and calprotectin test results. The red and green shading represent positive and negative results, respectively
| Sample number | Hip or knee | Histology* | Culture† | CRP‡ | Sinus tract§ | Purulence¶ | ICM** | Calprotectin†† |
|---|---|---|---|---|---|---|---|---|
| 1 | K | negative | negative | 13 | N | N | negative | negative |
| 2 | H | negative | negative | 40 | N | N | negative | negative |
| 3 | H | negative | negative | N/A | N | N | negative | negative |
| 4 | H | negative | negative | 2 | N | N | negative | positive |
| 5 | H | positive | negative | 47 | Y | Y | positive | positive |
| 6 | H | negative | negative | 2 | N | N | negative | negative |
| 7 | H | negative | negative | 6 | N | N | negative | negative |
| 8 | K | negative | negative | < 1 | N | N | negative | negative |
| 9 | H | negative | negative | 7 | N | N | negative | negative |
| 10 | H | negative | negative | 2 | N | N | negative | negative |
| 11 | H | negative | negative | 7 | N | N | negative | positive |
| 12 | H | negative | negative | 21 | N | N | negative | negative |
| 13 | H | positive | negative | 35 | Y | Y | positive | positive |
| 14 | H | negative | negative | < 1 | N | N | negative | positive |
| 15 | K | N/A | positive | 69 | Y | N | positive | positive |
| 16 | H | negative | negative | 1 | N | N | negative | positive |
| 17 | H | negative | negative | 4 | N | N | negative | negative |
| 18 | H | N/A | positive | 87 | N | Y | positive | negative |
| 19 | H | N/A | positive | 101 | N | Y | positive | positive |
| 20 | H | negative | negative | 1 | N | N | negative | negative |
| 21 | K | N/A | positive | 5 | N | N | positive | negative |
| 22 | H | N/A | positive | 25 | N | N | positive | positive |
| 23 | H | negative | negative | 10 | N | N | negative | negative |
| 24 | H | negative | negative | 5 | N | N | negative | negative |
| 25 | K | positive | positive | 286 | N | Y | positive | positive |
| 26 | H | N/A | positive | 44 | Y | N | positive | positive |
| 27 | H | negative | negative | 17 | N | N | negative | positive |
| 28 | H | negative | negative | 1 | N | N | negative | positive |
| 29 | K | negative | negative | 19 | N | N | negative | negative |
| 30 | H | positive | positive | 61 | N | Y | positive | positive |
| 31 | K | negative | negative | N/A | N | N | negative | negative |
| 32 | H | negative | negative | < 1 | N | N | negative | negative |
| 33 | H | negative | negative | 6 | N | N | negative | negative |
| 34 | K | positive | positive | 91 | N | N | positive | positive |
| 35 | H | negative | negative | 5 | N | N | negative | positive |
| 36 | H | negative | negative | 8 | N | N | negative | positive |
| 37 | H | positive | positive | 114 | N | Y | positive | positive |
| 38 | K | negative | negative | < 1 | N | N | negative | negative |
| 39 | H | negative | negative | 3 | N | N | negative | negative |
| 40 | H | negative | negative | 1 | N | N | negative | negative |
| 41 | K | negative | negative | 29 | N | N | negative | negative |
| 42 | H | N/A | positive | 18 | N | N | positive | positive |
| 43 | H | negative | negative | 1 | N | N | negative | negative |
| 44 | H | negative | negative | 4 | N | N | negative | negative |
| 45 | H | negative | negative | 2 | N | N | negative | negative |
| 46 | K | N/A | positive | 219 | N | N | positive | positive |
| 47 | H | negative | negative | < 1 | N | N | negative | negative |
| 48 | H | negative | positive | < 1 | N | N | positive | negative |
| 49 | H | negative | negative | 3 | N | N | negative | negative |
| 50 | K | negative | positive | 4 | N | N | positive | negative |
| 51 | H | positive | negative | 36 | N | Y | positive | positive |
| 52 | H | negative | negative | 2 | N | N | negative | negative |
| 53 | H | negative | negative | < 1 | N | N | negative | positive |
| 54 | K | negative | negative | 2 | N | N | negative | negative |
| 55 | K | N/A | positive | 31 | N | N | positive | positive |
| 56 | H | negative | positive | 30 | N | N | positive | negative |
| 57 | K | negative | positive | 1 | N | N | positive | negative |
| 58 | H | negative | negative | < 1 | N | N | negative | negative |
| 59 | H | N/A | positive | 49 | N | Y | positive | positive |
| 60 | K | N/A | positive | 67 | N | Y | positive | positive |
| 61 | H | negative | negative | 3 | N | N | negative | negative |
| 62 | H | N/A | positive | 284 | Y | Y | positive | positive |
| 63 | H | negative | negative | 15 | N | N | negative | negative |
| 64 | H | negative | negative | 4 | N | N | negative | negative |
| 65 | H | negative | negative | 3 | N | N | negative | positive |
| 66 | K | negative | negative | 6 | N | N | negative | negative |
| 67 | H | positive | negative | 15 | N | N | negative | positive |
| 68 | H | positive | negative | 3 | N | N | negative | negative |
| 69 | H | N/A | positive | 51 | Y | N | positive | positive |
-
*
Positive/negative for signs of inflammation by frozen section microscopy.
-
†
Positive/negative for two or more cultures of the same organism from periprosthetic tissue and fluid samples.
-
‡
Positive when serum CRP level ≥ 10 mg/l.
-
§
Y represents the presence of sinus tract.
-
¶
Y represents visible wound purulence.
-
**
Positive/negative according to scoring in Table I.
-
††
Positive when ≥ 14 mg/l synovial calprotectin measured by Lyfstone calprotectin test (Lyfstone AS, Tromsø, Norway).
-
ICM, International Consensus Meeting 2018 criteria; N/A, not available.
Table III.
Performance of calprotectin test on tested synovial fluid samples
| Lyfstone* test | Gold standard | |||||
|---|---|---|---|---|---|---|
| ICM | ICM-CR | |||||
| All samples (n = 69) | Hips (n = 52) | Knees (n = 17) | All samples (n = 69) | Hips (n = 52) | Knees (n = 17) | |
| Sensitivity, % (n/total n) (95% CI) | 75.00 (18/24) (53.29 to 90.23) | 80.00 (12/15) (51.91 to 95.67) | 66.67 (6/9) (29.93 to 92.51) | 94.74 (18/19) (73.97 to 99.87) | 92.31 (12/13) (63.97 to 99.81) | 100.00 (6/6) (54.07 to 100.00) |
| Specificity, % (n/total n) (95% CI) | 75.56 (34/45) (60.46 to 87.12) | 70.27 (26/37) (53.02 to 84.13) | 100.00 (8/8) (63.06 to 100.00) | 78.00 (39/50) (64.04 to 88.47) | 71.79 (28/39) (55.13 to 85.00) | 100.00 (11/11) (71.51 to 100.00) |
| PPV, % (n/total n) (95% CI) | 62.07 (18/29) (48.23 to 74.19) | 52.17 (12/23) (38.48 to 65.55) | 100.00 (6/6) (100.00 to 100.00) | 62.07 (18/29) (49.00 to 73.60) | 52.17 (12/23) (39.23 to 64.83) | 100.00 (6/6) (100.00 to 100.00) |
| NPV, % (n/total n) (95% CI) | 85.00 (34/40) (73.54 to 92.04) | 89.66 (26/29) (75.51 to 96.06) | 72.73 (8/11) (51.42 to 87.04) | 97.50 (39/40) (85.20 to 99.62) | 96.55 (28/29) (80.83 to 99.47) | 100.00 (11/11) (100.00 to 100.00) |
| Accuracy, % (n/total n) (95% CI) | 75.36 (52/69) (63.51 to 84.95) | 73.08 (38/52) (58.98 to 84.43) | 82.35 (14/17) (56.57 to 96.20) | 82.61 (57/69) (71.59 to 90.68) | 76.92 (40/52) (63.16 to 87.47) | 100.00 (17/17) (80.49 to 100.00) |
-
*
Lyfstone AS, Tromsø, Norway.
-
CI, confidence interval; ICM, International Consensus Meeting 2018 criteria; ICM-CR, International Consensus Meeting 2018 criteria with clinical review; NPV, negative predictive value; PPV, positive predictive value.
Fig. 2
Area under the receiver operating characteristic (ROC) curve (AUC) of the Lyfstone calprotectin test (Lyfstone AS, Tromsø, Norway) against International Consensus Meeting 2018 criteria (ICM) for infection. AUC = 0.779; p < 0.001.
A total of 24 cases were classed as infected (15 hips, nine knees) and 45 cases aseptic (37 hips, eight knees) according to ICM criteria. Of the infected cases, 21 were found to be positive by routine microbiology alone (two or more cultures were positive for the same organism). Two of the remaining cases were found to be positive by histology and elevated serum CRP, and one positive culture (Collinsella aerofaciens and ‘Diphtheroids’) and the remaining infected case were found positive by a combination of histology, CRP, and purulence (Table II).
In this infected group there were 18 true positive and six false negative results by calprotectin. The reported organisms in these false negative cases were Cutibacterium acnes (n = 3), Bacteroides fragilis (n = 1), Staphylococcus epidermidis (n = 1), and a polymicrobial infection of Pseudomonas aeruginosa with Corynebacterium tuberculostearicum (n = 1).
The remaining 45 cases were found to be aseptic by ICM criteria, and of these 34 were true negative and 11 were false positive according to the calprotectin test. Of the 11 false positive cases, one case was positive for inflammation by histology and elevated CRP and one was CRP positive without significant histology. The remaining nine false positive samples were negative for any ICM criteria tested.
In the hip revision surgery group (n = 52) the test had a sensitivity of 80.00% (12/15, 95% CI 51.91% to 95.67%) and specificity of 70.27% (26/37, 95% CI 53.02% to 84.13%). The PPV and NPV were 52.17% (12/23, 95% CI 38.48% to 65.55%) and 89.66% (26/29, 95% CI 75.51% to 96.06%), respectively, with an overall test accuracy of 73.08% (38/52, 95% CI 58.98% to 84.43%). All 11 false positive results in the study were hip revisions. In the knee revision surgery group (n = 17) the test was 66.67% sensitive (6/9, 95% CI 29.93% to 92.51%) and 100.00% specific (8/8, 95% CI 63.06% to 100.00%). With no false positive results, the PPV was 100.00% (6/6, 95% CI 100.00% to 100.00%) but the NPV was 72.73% (8/11, 95% CI 51.42% to 87.04%) (Table III). Overall test accuracy was higher than in the hip group at 82.35% (14/17, 95% CI 56.57% to 96.20%).
Lyfstone test performance compared to ICM-CR
Overall test accuracy compared to the ICM-CR was 82.61% (57/69, 95% CI 71.59% to 90.68%). Sensitivity and specificity were 94.74% (18/19, 95% CI 73.97% to 99.87%) and 78.00% (39/50, 95% CI 64.04% to 88.47%), respectively, with PPV of 62.07% (18/29, 95% CI 49.00% to 73.60%) and NPV of 97.50% (39/40, 95% CI 85.20% to 99.62%) (Table III). AUC compared to ICM-CR was 0.91 (95% CI 0.81 to 0.96) (Figure 3).
Fig. 3
Area under the receiver operating characteristic (ROC) curve (AUC) of Lyfstone calprotectin test (Lyfstone AS, Tromsø, Norway) against International Consensus Meeting 2018 with clinical review criteria (ICM-CR) for infection. AUC = 0.905; p < 0.001.
In the hip group, radiograph and medical record review revealed that five patients had metal-on-metal implants with evidence of an adverse reaction to metal debris. Additionally, two patients had severe metal staining of tissue caused by wear of the acetabular component following wear of the polyethylene liner. The remaining four false positive cases had aseptic loosening listed as the initial indication for operation. Review of preoperative radiographs and operation notes revealed pre- and intraoperative evidence of osteolysis in three out of four cases, to proximal femur or acetabulum (Table IV).
Table IV.
Paprosky femoral and acetabular classification of patients diagnosed with aseptic loosening and with high calprotectin levels reported
| Sample number | Femur | Acetabulum |
|---|---|---|
| 4 | IIIB | I |
| 27 | I | IIIA |
| 28 | II | IIB |
One-year follow-up of the remaining unresolved case (patient 35) revealed that they were still experiencing pain in their joint and their CRP had risen from 5 mg/l to 19 mg/l, indicating possible infection. However, this remains under investigation by the treating teams.
A total of 53 patients remained once metallosis and severe osteolysis cases had been removed from the dataset. This improved the specificity, PPV, and overall accuracy of the test both against ICM and ICM-CR (Table V).
Table V.
Performance of calprotectin test on tested synovial fluid samples, excluding metallosis and severe osteolysis cases
| Lyfstone* test | Gold standard | |
|---|---|---|
| ICM (n = 53)† | ICM-CR‡ | |
| Sensitivity, % (95% CI) | 71.43 (47.82 to 88.72) | 93.75 (69.77 to 99.84) |
| Specificity, % (95% CI) | 96.88 (83.78 to 99.92) | 97.30 (85.84 to 99.93) |
| PPV, % (95% CI) | 93.75 (68.14 to 99.06) | 93.75 (68.36 to 99.05) |
| NPV, % (95% CI) | 83.78 (72.37 to 91.06) | 93.70 (84.36 to 99.59) |
| Accuracy, % (95% CI) | 86.79 (74.66 to 94.52) | 96.23 (87.02 to 99.54) |
-
*
Lyfstone AS, Tromsø, Norway.
-
†
Calprotectin result against International Consensus Meeting criteria (n = 53) for infection excluding all samples associated with metallosis and severe osteolysis.
-
‡
Calprotectin result against International Consensus Meeting criteria (n = 53) for infection with discrepant samples investigated by clinical follow-up excluding all samples associated with metallosis and severe osteolysis.
-
CI, confidence interval; ICM, International Consensus Meeting 2018 criteria; ICM-CR, International Consensus Meeting 2018 criteria with clinical review; NPV, negative predictive value; PPV, positive predictive value.
The three false negative results within the hip subset were all ICM-positive by culture alone (two or more cultures of the same organism). Patient 18 was culture-positive for B. fragilis, isolated from all cultured tissue samples, along with a high CRP and noted joint purulence. Calprotectin was recorded at 13.36 mg/l, which falls just below the threshold value for a positive result (≥ 14 mg/l) (Supplementary Table i). Sample 48 was culture-positive for C. acnes but negative by both histology and CRP. The calprotectin test was negative with 0.0 mg/l. The initial indication for operation for this case was aseptic loosening and a single-stage revision was performed. The C. acnes report was dismissed as contamination and the patient was not treated for infection. One-year follow-up of the patient showed no signs of infection indicating that the calprotectin result correlated with the clinical findings. Sample 56 was a similar case of an aseptic loosening revision yielding significant culture results (P. aeruginosa and C. tuberculostearicum) but with no detectable synovial fluid calprotectin. The patient had a single-stage revision and upon a two-month follow-up showed no sign of infection, indicating a true negative calprotectin result.
The clinical review (ICM-CR) increases the sensitivity of the Lyfstone test from 75.00% (18/24, 95% CI 53.29% to 90.23%) to 94.74% (18/19, 95% CI 73.97% to 99.87%) (Table III).
In the knee group, three false negative results were recorded; two cases of C. acnes and one case reporting S. epidermidis. Both C. acnes cases (samples 50 and 57) were patients displaying no preoperative indication of infection and were treated as aseptic loosening. The clinical review found no complications after patient follow-up, suggesting that both cases were aseptic and that the calprotectin results were correct. Sample 21 (S. epidermidis) was taken during the second stage of revision surgery performed one year after the first-stage operation. Clinical review revealed that the patient was originally treated for gross infection. At the time of the second-stage revision, the patient showed no overt signs of infection, so treatment progressed to implantation of the prosthesis. Postoperative microbiology grew a culture of S. epidermidis from three out of six samples taken at the time of surgery. At the one-year follow-up the patient presented no sign of infection, confirming the calprotectin result to be correct thus far. Overall, the Lyfstone test was 100% accurate for the diagnosis of PJI in knee samples (n = 17; Tables II and III).
Discussion
PJI is a serious complication of arthroplasty with associated patient morbidity and a substantial economical burden of treatment.23,24 Microbiological culture techniques are slow and insensitive8,9 and although new diagnostic tests are being developed, none of the current methods are capable of reliably diagnosing PJI.5,25
Rapid biomarker-based tests have the potential to assist clinicians in making a preoperative diagnosis. Early differentiation of PJI from aseptic loosening may reduce the number of unnecessary two-stage revisions performed on patients with a presumptive diagnosis of low-grade infection.
The Synovasure test (Zimmer Biomet, Warsaw, Indiana, USA) functions very similarly to the Lyfstone calprotectin test, measuring another neutrophil-released antimicrobial peptide, α-defensin. Studies using Synovasure have shown varying results with sensitivities ranging from 67.0% to 100.0% and specificities ranging from 82.4% to 100.0% against the MSIS/ICM criteria.26–35 A systematic review summarizing the results of seven prospective trials found that the Synovasure lateral flow test had a mean sensitivity of 85% and a mean specificity of 90%.36
Recent studies on using calprotectin to diagnose PJI either using an enzyme-linked immunosorbent assay (ELISA) method or an off-label lateral flow assay for measuring faecal calprotectin have shown similar diagnostic performance. Wouthuyzen-Bakker et al17,37 applied a faecal calprotectin test to PJI, demonstrating a sensitivity of 89% and a specificity of 90% in a pilot study37 and 86.7% and 91.6% in a follow-on study.17 A more recent study by Salari et al38 reported a sensitivity of 100% and a specificity of 95% utilizing an ELISA-based calprotectin test. The Lyfstone calprotectin lateral flow test piloted here demonstrates a sensitivity of 75.00% and a specificity of 75.56% using similar gold standard diagnostic criteria. Our study is of a comparable size (69 samples compared to 6137, 52,17 and 7638) and featured a similar prevalence of infected samples (35% compared to 31%37, 29%,17 and 37%38), but the sensitivity and specificity of the lateral flow assay is lower than previous calprotectin-based studies have shown. However, similar differences in performance have been reported between the Synovasure α-defensin lateral flow test and the α-defensin ELISA method.39
Clinical review of discordant cases revealed that the low specificity of the calprotectin lateral flow test was associated with metallosis (7/11 false positive cases). Removing these samples increases the specificity from 75.56% to 83.78%. The use of biomarker-based diagnostic tests is known to be unreliable in patients with metal-on-metal implants as these can cause gross inflammation producing false positive results.40,41 Severe osteolysis can cause false positive results for the same reason and removing patients with severe osteolysis (n = 3) from the analysis further increased specificity to 96.88%, which is more in line with other published calprotectin studies (notably Salari et al38 excluded patients with metal-on-metal implants).
The clinical review also revealed that five out of six false negative results were in samples where the organisms detected by culture could be considered contaminants. Three of the six false negative samples were positive for C. acnes by culture. This organism is a common laboratory contaminant and not typically considered a cause of PJI when isolated from hip or knee tissues.42–44 Only one false negative case was deemed a genuine infection by clinical review (case 18) in which the calprotectin result was very close to positive (13.36 mg/l, < 1 mg/l below the positive cutoff). If future optimization of the test sets a lower threshold value for infection, then this case would have been classified as positive. Such a readjustment of the calprotectin cutoff values may improve test performance in detecting low-grade infections.
The sensitivity and specificity of 93.75% and 97.30% after clinical review are consistent with other studies on calprotectin. Differences in inclusion criteria, sample collection, and microbiology techniques applied in other studies may explain differences in performance observed. Observed test accuracy was different for hip and knee revisions. False positive results were only observed in hip revisions, likely affected by the absence of metal-on-metal reactions in the knee revision group. While the test accuracy for knee revisions compared to ICM-CR was 100%, the sample size is small (17 revisions, eight positive for infection) and the 95% CIs are too wide (sensitivity CI 54.07% to 100.00%; specificity CI 71.51% to 100.00%) to draw conclusions with high confidence. As such, more data are required to confirm these findings.
The study was limited by the use of retrospective samples. However, by testing all eligible samples in the Biorepository we believe that we did not introduce any sample selection bias. The relatively small sample size has resulted in wide 95% CIs throughout, hence larger studies are required to confirm study findings. Another limitation was that the samples had been frozen for storage before testing and the freeze-thaw process may have resulted in leucocyte cell lysis and an increase in calprotectin. As the test is validated for use on fresh synovial fluid samples, the recommended cutoffs may not necessarily be appropriate for our samples. It is worth noting that blood contamination of the samples had no impact on the accuracy of the test results, something the Synovasure lateral flow test warns may lead to false negative results.45
In conclusion, the Lyfstone calprotectin lateral flow test shows good potential as a rapid diagnostic for infected arthroplasty cases. The high NPV makes this a potentially powerful rule-out test for infection in suspected PJI. However, larger prospective studies, using consecutive samples from patients undergoing revision of all synovial joint arthroplasties, are required to more accurately define the test’s diagnostic performance.
Open access
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-Commercial No Derivatives (CC BY-NC-ND 4.0) licence, which permits the copying and redistribution of the work only, and provided the original author and source are credted. See https://creativecommons.org/licenses/by-nc-nd/4.0/.
Acknowledgements
I. McNamara and J. O’Grady contributed equally to this study.
Supplementary Material
Table showing additional patient data including number of samples organisms isolated from, and calprotectin test scores by risk category and quantification in mg/l, sex, age, body mass index (BMI), location of operation, indication for operation, procedure undertaken, and any relevant past medical history.
Funding statement
J. Mikalsen reports payment from Lyfstone AS during the course of this study, and a pending International Patent Application (No. PCT/EP2018/053044).
J. O’Grady and J. Wain report accommodation expenses paid by Lyfstone AS for a project meeting in Tromsø, Norway in relation to this study.
A. J. Trotter reports travel expenses paid by Lyfstone AS for a conference related to this study. I. McNamara reports a grant and personal consultancy fees from Lyfstone AS for advice on diagnosis and treatment of prosthetic joint infection in relation to this study.
This paper presents independent research funded by Orthopaedic Research UK (ORUK – grant number 526) and the Biotechnology and Biological Sciences Research Council (BBSRC) Institute Strategic Programme Microbes in the Food Chain BB/R012504/1 and its constituent projects BBS/E/F/000PR10348 and BBS/E/F/000PR10349 (JOG).
The author or one or more of the authors have received or will receive benefits for personal or professional use from a commercial party related directly or indirectly to the subject of this article.
Conflict of interest statement
None declared
Ethical review statement
Ethical approval was provided by University of East Anglia, Faculty of Medicine and Health Sciences Research Ethics Committee, project reference: 2016/1721SE. Informed consent was obtained from all patients included in the study.
Follow us @BoneJointRes
References
1. No authors listed. HQIP. National Joint Registry 15th Annual Report. 2018. https://www.hqip.org.uk/resource/national-joint-registry-15th-annual-report-2018/ (date last accessed 3 February 2020). Google Scholar
2. Tande AJ , Patel R . Prosthetic joint infection. Clin Microbiol Rev. 2014;27(2):302-345.CrossrefPubMed Google Scholar
3. Pérez-Prieto D , Portillo ME , Puig-Verdié L , et al. C-reactive protein may misdiagnose prosthetic joint infections, particularly chronic and low-grade infections. Int Orthop. 2017;41(7):1315-1319.CrossrefPubMed Google Scholar
4. Siedlecki C , Beaufils P , Lemaire B , Pujol N . Complications and cost of single-stage vs. two-stage bilateral unicompartmental knee arthroplasty: A case-control study. Orthop Traumatol Surg Res. 2018;104(7):949-953.CrossrefPubMed Google Scholar
5. Parvizi J , Zmistowski B , Berbari EF , et al. New definition for periprosthetic joint infection: from the Workgroup of the Musculoskeletal Infection Society. Clin Orthop Relat Res. 2011;469(11):2992-2994.CrossrefPubMed Google Scholar
6. Osmon DR , Berbari EF , Berendt AR et al. ; Infectious Diseases Society of America. Executive summary: diagnosis and management of prosthetic joint infection: clinical practice guidelines by the Infectious Diseases Society of America. Clin Infect Dis. 2013;56(1):1-10.CrossrefPubMed Google Scholar
7. Parvizi J , Tan TL , Goswami K , et al. The 2018 Definition of Periprosthetic Hip and Knee Infection: An Evidence-Based and Validated Criteria. J Arthroplasty. 2018;33(5):1309-1314.e2.CrossrefPubMed Google Scholar
8. Boyle KK , Wood S , Tarity TD . Low-Virulence Organisms and Periprosthetic Joint Infection-Biofilm Considerations of These Organisms. Curr Rev Musculoskelet Med. 2018;11(3):409-419.CrossrefPubMed Google Scholar
9. Parvizi J , Erkocak OF , Della Valle CJ . Culture-negative periprosthetic joint infection. J Bone Joint Surg Am. 2014;96-A(5):430-436.CrossrefPubMed Google Scholar
10. Deirmengian C , Kardos K , Kilmartin P , Cameron A , Schiller K , Parvizi J . Combined measurement of synovial fluid α-Defensin and C-reactive protein levels: highly accurate for diagnosing periprosthetic joint infection. J Bone Joint Surg Am. 2014;96-A(17):1439-1445.CrossrefPubMed Google Scholar
11. Sousa R , Serrano P , Gomes Dias J , Oliveira JC , Oliveira A . Improving the accuracy of synovial fluid analysis in the diagnosis of prosthetic joint infection with simple and inexpensive biomarkers: C-reactive protein and adenosine deaminase. Bone Joint J. 2017;99-B(3):351-357.CrossrefPubMed Google Scholar
12. Tarabichi M , Fleischman AN , Shahi A , Tian S , Parvizi J . Interpretation of Leukocyte Esterase for the Detection of Periprosthetic Joint Infection Based on Serologic Markers. J Arthroplasty. 2017;32(9S):S97-S100.e1.CrossrefPubMed Google Scholar
13. Stríz I , Trebichavský I . Calprotectin - a pleiotropic molecule in acute and chronic inflammation. Physiol Res. 2004;53(3):245-253.PubMed Google Scholar
14. Witter AR , Okunnu BM , Berg RE . The Essential Role of Neutrophils during Infection with the Intracellular Bacterial Pathogen Listeria monocytogenes. J Immunol. 2016;197(5):1557-1565.CrossrefPubMed Google Scholar
15. van Rheenen PF , Van de Vijver E , Fidler V . Faecal calprotectin for screening of patients with suspected inflammatory bowel disease: diagnostic meta-analysis. BMJ. 2010;341:c3369.CrossrefPubMed Google Scholar
16. Abildtrup M , Kingsley GH , Scott DL . Calprotectin as a biomarker for rheumatoid arthritis: a systematic review. J Rheumatol. 2015;42(5):760-770.CrossrefPubMed Google Scholar
17. Wouthuyzen-Bakker M , Ploegmakers JJW , Ottink K , et al. Synovial Calprotectin: An Inexpensive Biomarker to Exclude a Chronic Prosthetic Joint Infection. J Arthroplasty. 2018;33(4):1149-1153.CrossrefPubMed Google Scholar
18. Paprosky WG , Perona PG , Lawrence JM . Acetabular defect classification and surgical reconstruction in revision arthroplasty. A 6-year follow-up evaluation. J Arthroplasty. 1994;9(1):33-44.CrossrefPubMed Google Scholar
19. No authors listed. LYFCLP005 Lateral Flow Test Kit Instructions For Use. Lyfstone AS. Google Scholar
20. No authors listed. UK Standards for Microbiology Investigations: Investigation of orthopaedic implant associated infections. Public Health England. 2016. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/504319/B_44i2.pdf (date last accessed 27 February 2020). Google Scholar
21. No authors listed. UKAS Medical ISO 8405. Service UKA. https://www.ukas.com/wp-content/uploads/schedule_uploads/00007/8405%20Medical%20Single.pdf (date last accessed 27 February 2020). Google Scholar
22. No authors listed. UKAS Medical ISO 10295. Service UKA. 2018. https://www.ukas.com/wp-content/uploads/schedule_uploads/00007/10295%20Medical%20Single.pdf (date last accessed 28 February 2020). Google Scholar
23. Berend KR , Lombardi AV Jr , Morris MJ , Bergeson AG , Adams JB , Sneller MA . Two-stage treatment of hip periprosthetic joint infection is associated with a high rate of infection control but high mortality. Clin Orthop Relat Res. 2013;471(2):510-518.CrossrefPubMed Google Scholar
24. Barrack RL , Burnett RS , Sharkey P , Parvizi J . Diagnosing an infection: an unsolved problem. Orthopedics. 2007;30(9):777-778.CrossrefPubMed Google Scholar
25. Fernández-Sampedro M , Fariñas-Alvarez C , Garces-Zarzalejo C , et al. Accuracy of different diagnostic tests for early, delayed and late prosthetic joint infection. BMC Infect Dis. 2017;17(1):592.CrossrefPubMed Google Scholar
26. Berger P , Van Cauter M , Driesen R , Neyt J , Cornu O , Bellemans J . Diagnosis of prosthetic joint infection with alpha-defensin using a lateral flow device: a multicentre study. Bone Joint J. 2017;99-B(9):1176-1182.CrossrefPubMed Google Scholar
27. Bingham J , Clarke H , Spangehl M , Schwartz A , Beauchamp C , Goldberg B . The alpha defensin-1 biomarker assay can be used to evaluate the potentially infected total joint arthroplasty. Clin Orthop Relat Res. 2014;472(12):4006-4009.CrossrefPubMed Google Scholar
28. Eriksson HK , Nordström J , Gabrysch K , Hailer NP , Lazarinis S . Does the Alpha-defensin Immunoassay or the Lateral Flow Test Have Better Diagnostic Value for Periprosthetic Joint Infection? A Meta-analysis. Clin Orthop Relat Res. 2018;476(5):1065-1072.CrossrefPubMed Google Scholar
29. Frangiamore SJ , Gajewski ND , Saleh A , Farias-Kovac M , Barsoum WK , Higuera CA . α-Defensin Accuracy to Diagnose Periprosthetic Joint Infection-Best Available Test?J Arthroplasty. 2016;31(2):456-460.CrossrefPubMed Google Scholar
30. Gehrke T , Lausmann C , Citak M , Bonanzinga T , Frommelt L , Zahar A . The Accuracy of the Alpha Defensin Lateral Flow Device for Diagnosis of Periprosthetic Joint Infection: Comparison with a Gold Standard. J Bone Joint Surg Am. 2018;100-A(1):42-48.CrossrefPubMed Google Scholar
31. Kasparek MF , Kasparek M , Boettner F , Faschingbauer M , Hahne J , Dominkus M . Intraoperative Diagnosis of Periprosthetic Joint Infection Using a Novel Alpha-Defensin Lateral Flow Assay. J Arthroplasty. 2016;31(12):2871-2874.CrossrefPubMed Google Scholar
32. Riccio G , Cavagnaro L , Akkouche W , Carrega G , Felli L , Burastero G . Qualitative Alpha-defensin Versus The Main Available Tests For The Diagnosis Of Periprosthetic Joint Infection: Best Predictor Test?J Bone Jt Infect. 2018;3(3):156-164.CrossrefPubMed Google Scholar
33. Sigmund IK , Yermak K , Perka C , Trampuz A , Renz N . Is the Enzyme-linked Immunosorbent Assay More Accurate Than the Lateral Flow Alpha Defensin Test for Diagnosing Periprosthetic Joint Infection?Clin Orthop Relat Res. 2018;476(8):1645-1654.CrossrefPubMed Google Scholar
34. Stone WZ , Gray CF , Parvataneni HK , et al. Clinical Evaluation of Synovial Alpha Defensin and Synovial C-Reactive Protein in the Diagnosis of Periprosthetic Joint Infection. J Bone Joint Surg Am. 2018;100-A(14):1184-1190.CrossrefPubMed Google Scholar
35. Kleiss S , Jandl NM , Novo de Oliveira A , Ruther W , Niemeier A . Diagnostic accuracy of alpha-defensin enzyme-linked immunosorbent assay in the clinical evaluation of painful hip and knee arthroplasty with possible prosthetic joint infection: a prospective study of 202 cases. Bone Joint J. 2019;101-B(8):970-977.CrossrefPubMed Google Scholar
36. Marson BA , Deshmukh SR , Grindlay DJC , Scammell BE . Alpha-defensin and the Synovasure lateral flow device for the diagnosis of prosthetic joint infection: a systematic review and meta-analysis. Bone Joint J. 2018;100-B(6):703-711.CrossrefPubMed Google Scholar
37. Wouthuyzen-Bakker M , Ploegmakers JJW , Kampinga GA , Wagenmakers-Huizenga L , Jutte PC , Muller Kobold AC . Synovial calprotectin: a potential biomarker to exclude a prosthetic joint infection. Bone Joint J. 2017;99-B(5):660-665.CrossrefPubMed Google Scholar
38. Salari P , Grassi M , Cinti B , Onori N , Gigante A . Synovial Fluid Calprotectin for the Preoperative Diagnosis of Chronic Periprosthetic Joint Infection. J Arthroplasty. 2019.CrossrefPubMed Google Scholar
39. Suen K , Keeka M , Ailabouni R , Tran P . Synovasure ‘quick test’ is not as accurate as the laboratory-based alpha-defensin immunoassay: a systematic review and meta-analysis. Bone Joint J. 2018;100-B(1):66-72. Google Scholar
40. Drummond J , Tran P , Fary C . Metal-on-Metal Hip Arthroplasty: A Review of Adverse Reactions and Patient Management. J Funct Biomater. 2015;6(3):486-499.CrossrefPubMed Google Scholar
41. Bonanzinga T , Zahar A , Dütsch M , Lausmann C , Kendoff D , Gehrke T . How Reliable Is the Alpha-defensin Immunoassay Test for Diagnosing Periprosthetic Joint Infection? A Prospective Study. Clin Orthop Relat Res. 2017;475(2):408-415.CrossrefPubMed Google Scholar
42. Both A , Klatte TO , Lübke A , et al. Growth of Cutibacterium acnes is common on osteosynthesis material of the shoulder in patients without signs of infection. Acta Orthop. 2018;89(5):580-584.CrossrefPubMed Google Scholar
43. Levy PY , Fenollar F , Stein A , et al. Propionibacterium acnes postoperative shoulder arthritis: an emerging clinical entity. Clin Infect Dis. 2008;46(12):1884-1886.CrossrefPubMed Google Scholar
44. Patel A , Calfee RP , Plante M , Fischer SA , Green A . Propionibacterium acnes colonization of the human shoulder. J Shoulder Elbow Surg. 2009;18(6):897-902.CrossrefPubMed Google Scholar
45. No authors listed. Zimmer Biomet. Diagnostics C. Synovasure Alpha Defensin Lateral Flow Test Kit: Instructions For Use. https://cddiagnostics.com/instructions/PDF/LF%20Test%20Kit%20IFUs/EN_M40004B_V6_Synovasure_Alpha_Defensin_Lateral_Flow_IFU.pdf (date last accessed 3 February 2020). Google Scholar
