Responsiveness and ceiling effects of the Forgotten Joint Score-12 following total hip arthroplasty

Article focus

• The use of outcome measures to qualify success following total hip arthroplasty; specifically to assess the responsiveness of the Forgotten Joint Score – 12 in a United Kingdom population.

Key messages

• The FJS-12 is more responsive to change between six and 12 months after total hip arthroplasty and demonstrates half the ceiling effects of the Oxford Hip Score.

• Use of a more sensitive score has material implications in detecting differences in patient outcome over time.

• Use of an outcome measure with greater measurement range and precision allows for more specific studies with smaller numbers of participants, which has significant benefits for researchers and funders, allowing for quicker and cheaper studies.

Strengths and limitations

• This study is the first specific evaluation of the methodological qualities of the FJS-12 in a United Kingdom or European population of hip arthroplasty patients.

• The FJS-12 was described as a post-operative outcome score, however the use of pre-operative data, would offer additional perspective in terms of assessing change in score following surgery.

Introduction

Patient-reported outcome measures (PROMs) are a key parameter for assessing outcome following total hip arthroplasty. PROMs are widely used to power clinical studies and increasingly complement implant survivorship in national joint registries, providing a measure of a patient’s pain and physical function.^1,2

There is a plethora of potential patient outcome tools that can be employed, which, though similar, offer a unique perspective on patient outcome. There is no consensus as to which scores ‘should’ be used, and researchers typically revert to the most commonly reported scores such as (for hip arthroplasty) the Oxford Hip Score (OHS)³ or Harris Hip Score⁴

PROM questionnaires differ substantially in their ability to capture change following joint replacement surgery.^5,6 The sensitivity of a score to detect change is critical in determining the level of function a patient has achieved. Ceiling effects in a scoring system can hide differences when patients already record the maximum possible score and cannot improve on that score. For example, in a score that simply assesses whether patients can walk for more than 30 minutes without pain, a 75-year-old patient able to walk for just 30 minutes at a normal pace would have the same score as a 45-year-old patient who has returned to running marathons.

When designing clinical outcome studies, the responsiveness of a score is of key importance in determining which score is used to power the study, as this materially affects the sample size required. Responsive scores are required as the symptom burden and range of functional impairments define whether ceiling effects will reduce the discriminatory power of the employed PROM. In terms of assessing the outcome of joint arthroplasty, a measure should have a measurement range that is responsive across the patient’s surgical pathway, including pre-operative and longitudinal post-operative outcomes.

A new joint-specific score, the Forgotten Joint Score-12 (FJS-12),⁷ was introduced in 2012 to asses a novel construct, ‘joint awareness’. The authors of this score suggested that the new construct would be more responsive to higher level functional outcomes after joint arthroplasty, because being able to ‘forget’ the joint during daily life is perhaps the ultimate goal following joint replacement surgery.

The objective of this study was to assess the responsiveness and ceiling effects of the Forgotten Joint Score-12 and compare these measures with that of the more widely used Oxford Hip Score in patients six and 12 months following primary total hip arthroplasty.

Patients and Methods

Results

Sensitivity to change over time

The mean OHS improved from 40.3 (sd 7.9) at six months to 41.9 (sd 7.2) at 12 months. Mean FJS-12 improved from 56.8 (sd 30.1) at six months to 62.1 (sd 29.0) at 12 months (Table II). At six months, 15.5% of the patients reached the best possible score (48 points) on the OHS and 8.3% obtained the best score (100 points) on the FJS-12. At 12 months, this percentage increased to 20.8% for the OHS and to 10.4% for the FJS-12 (Figs 1 and 2).

Fig. 1

Boxplot of the Oxford Hip Score at six and 12 months post-operatively

Fig. 2

Boxplot of the Forgotten Joint Score-12 at six and 12 months post-operatively

In terms of the effect size (Cohen’s d), the change was d = 0.10 for the OHS and d = 0.17 for the FJS-12. In a power analysis for the comparison of two time points (t-test for dependent sample, power = 0.80, alpha = 0.05, two-sided), the required sample size to detect a difference between six and 12 months is 787 patients for the OHS and 262 patients for the FJS-12.

Discussion

These findings suggest that the FJS-12 is more responsive to change between six and 12 months after total hip arthroplasty than is the OHS. The measured ceiling effect for the OHS was twice that of the FJS-12.

A ceiling effect means that several patients score the highest possible score achievable with the employed test. A ceiling effect occurs when the test items are not challenging enough for a group of individuals because the test has a limited number of difficult items or even an inappropriate item selection.¹¹ This will lead to a shortcoming in the discriminative ability of the test to detect clinically relevant changes so that if the patient continues to improve, the test does not capture that improvement. A ceiling effect of 15% is considered the maximum acceptable cut-off value.¹²

We found minimal difference in patient post-operative outcome as measured with the OHS between six and 12 months post-hip arthroplasty, however, the FJS-12 reported a larger change between six and 12 months, perhaps more accurately reflecting the changing perception of the patient as to the impact of the THA on their quality of life. The subgroup comparisons further highlight this difference, where the FJS-12 offered greater sensitivity to change than did the OHS in younger patients (under 70), healthier patients and males, i.e. those most likely to achieve high scores.^13,14

Outcome tools to assess joint arthroplasty first appeared in the late 1960s. When the Harris Hip Score was designed (in 1969), its content validity may well have been sufficient for the surgical case-mix of that time. However, more than 40 years later primary hip arthroplasty patients and the lifestyle of that age group has changed considerably. Wamper et al¹⁵ found a relevant ceiling effect after hip arthroplasty in 31 out of 59 studies in their meta-analysis of studies reporting Harris Hip Score data. Similarly, the OHS is an ageing tool developed in the 1990s. It has been well validated, but is increasingly understood to suffer from ceiling effects.^9,12 A major difference between the FJS-12 and the OHS is the population for which they were designed. The OHS was developed in a pre-operative population³ and reflects the symptom state at this time. The FJS-12, in contrast, was designed in a post-operative population and reflects the symptom state following surgery. This, in principle, should allow more measurement variation for higher level function and display a lesser ceiling effect, as is seen in Figures 1 and 2.

The Forgotten Joint Score was formally introduced in 2012⁷ and various groups have described the measurement properties of the FJS-12 in knee arthroplasty;^6,8 however, there is a lack of literature as to the score measurement properties following total hip arthroplasty. The original paper described similar ceiling and floor effects to the current data in the United Kingdom (9.3 and 3.3 %, respectively) in a Swiss population, although at a longer mean follow-up period of 3 years. Matsumoto et al¹⁶ recently validated a translated version of the FJS-12 in a Japanese hip arthroplasty population of 108 patients. These authors reported a single post-operative time point analysis comparing the FJS-12 to WOMAC and JEHQ scores at a mean follow-up of 30 months (with a wide follow-up time point range of one to 180 months). The authors reported a lower ceiling effect in the FJS-12 compared with the other measures, but did not report the percentage of patients achieving the top score, making direct comparisons difficult. Homma et al¹⁷ included the FJS-12 as an outcome metric in a study evaluating the effect of the surgical approach on lateral femoral cutaneous nerve injury rates, again in a Japanese population, of 122 patients. Measurement properties of the score were not evaluated, but the outcome scores presented for patients (post-operative FJS-12 of 64.3) are similar to those which we report in this United Kingdom population (62.1) and to those which Behrend et al⁷ reported in the original Swiss population (59.8).

The strengths of this study include the first presentation of linked post-operative FJS-12 outcome data at six and 12 months following total hip arthroplasty. The current study is also the first specific evaluation of the methodological qualities of this scoring system in a United Kingdom or European population of hip arthroplasty patients (aside from the original Swiss population). Limitations of this work include the restricted time points available for analysis. It would be useful to identify responsiveness over a longer post-operative timeframe, though 12 months is the typical outcome timeframe reported in the orthopaedic literature. In addition, the fact that the outcome scores presented herein are similar to those reported in differing populations at three years suggests that there is little further change in symptomology, although detailed investigation is required to confirm this. Furthermore, the FJS-12 was described as a post-operative outcome score and can be contrasted against healthy control population scores to offer context. Use of pre-operative data, however, would offer additional perspective in terms of assessing change in score following surgery. We suggest this as an avenue for future research.

Use of a more sensitive score that is not restricted by ceiling effects has material implications, both in detecting differences in patient outcome/function over time, and in the numbers of patients needed in a study to detect a difference in outcome between groups. The results we report herein suggest that around half the number of participants would be required to power a superiority design randomised trial when using the FJS-12 compared with the OHS. There is seemingly little difference in the effect sizes we report (0.10 and 0.17) but this translates as very different sample size requirements in terms of numbers of participants; this is because the association between effect size and sample size is essentially exponential. Use of an outcome measure with greater measurement range and precision allows for more specific studies with smaller numbers of participants, which has significant benefits for researchers and funders, allowing for quicker and cheaper studies.

Our findings suggest that the FJS-12 is more responsive to change between six and 12 months after total hip arthroplasty than is the OHS. The measured ceiling effect for the OHS was twice that of the FJS-12. The difference in effect size of change results in substantial differences in required sample size if aiming to detect change between these two time points. This has important implications for powering clinical trials with patient reported outcomes as the primary outcome.