What outcomes have been reported on patients following open lower limb fracture, and how have they been measured?

Article focus

• What outcomes have been reported in the academic literature on adult patients following open lower limb fracture over ten years?

• How have these outcomes been measured?

Key messages

• Existing literature on open lower limb fractures is of largely low-quality case series. It is hampered by outcome heterogeneity, wide variation in outcome definitions, use of multiple measurement tools, and infrequent collection of patient-reported outcome measures.

• This systematic review highlights significant inadequacies in outcome reporting in the current literature on open lower limb fracture, both in terms of outcome heterogeneity and a failure to report the most important outcomes from a patient perspective.

Strengths and limitations

• This systematic review provides a comprehensive overview of all outcomes and measurement instruments reported in the literature on open lower limb fracture over ten years.

• A search strategy updated after two years demonstrated data saturation in the number of outcomes reported on patients recovering from open lower limb fracture.

Introduction

Open lower limb fracture is a life-altering injury affecting 11.5 per 100,000 adults each year,¹ causing significant morbidity and mortality for patients and high resource demand on the UK trauma infrastructure.^1-3 Treatment is complex and multifaceted, such that an optimal treatment pathway for these devastating injuries remains unclear. Inconsistency in outcome reporting and poor-quality studies hamper attempts to evaluate new interventions to improve function, recovery experience, and investigation into the effects of regional variation of treatment on recovery.

Advancements in medical and surgical practice have expanded the proportion of severely injured limbs that are now amenable to limb salvage and reconstruction.⁴ Patient-centred care is a guiding principle of modern medical practice. The outcomes used in orthopaedic trauma research should reflect this, e.g. use of patient-reported outcome measures (PROMs). However, this does not appear to be reflected in the literature.^4,5 Scoping literature searches demonstrate that most studies reporting outcomes on patients recovering from open lower limb fracture tend to measure outcomes indicative of surgical success, e.g. objective clinical measures including bone union, alignment, and infection status. These preliminary findings are supported by Morris et al,⁵ who demonstrate considerable outcome heterogeneity reported in studies on closed tibial fractures. Failure to measure patient-reported outcomes will continue to propagate a discord between the surgeon and patient perceptions of successful treatment. Inconsistency in outcome reporting in the literature hampers comparative research and the ability to produce high-level evidence to drive improvements in patient care.

Methods

This systematic review was registered on the Prospective Register of Systematic Reviews (PROSPERO CRD42015024219) and reported with the aid of the PRISMA Statement.

In a clinical trial, an outcome refers to what is being measured or reported by participants, which is used to examine the effect of the intervention. An outcome measurement instrument refers to how the outcome is measured. It is a tool to measure the quality or quantity of an outcome.

Results

A total of 26,988 articles were identified from a primary and updated search. After excluding duplicates and limiting date ranges, 4,143 articles were shortlisted for the title and abstract review. Article screening identified 606 articles for full-text review; the article screening conflict rate between authors was less than 5%. Following a full-text review, 532 articles were included detailing outcomes reported on 429,076 participants for inclusion. Comparing duplicate full-text review and data extraction by HC and COA to ALA revealed no conflicts in studies included/excluded, and no significant differences in data fields extracted. See the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flowchart in Figure 1 for the study identification process, and reasons for article rejection at full-text review.

Fig. 1

Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flow diagram. OLLF, open lower limb fracture.

There was a relatively even spread in the number of studies published between 2009 and 2019. Case series and cohort studies represented 53% and 29% of all included studies, respectively, and RCTs represented 5% (29 individual studies) (see Table II). The majority of included studies reported outcomes on between three and 50 participants (52% of included studies). Studies were mostly conducted from the USA, UK, and China (23%, 14%, and 13% of articles, respectively).

Discussion

There is widespread heterogeneity in the outcomes reported across all outcome domains, where large numbers of heterogeneous outcome terms were grouped under the same standardized outcome heading. Many different definitions were used to define the same outcomes. This is highlighted in the musculoskeletal and connective tissue outcome domain, where 239 unique outcome definitions were given with no standard texts used between studies to define the same outcomes. These findings illustrate widespread inconsistency in outcome reporting and little-to-no standardization in outcome definitions.

Objective clinical measures such as bone union and infection represented most outcomes identified, indicating a preference by authors for outcomes relevant particularly to surgeons. This reflects reporting on outcomes important to surgeons collected as part of routine practice due to their ease of measurement, despite guidance and research trends advocating for the increased measurement of life impact outcomes in the form of PROMs.⁶ Publication of the ‘Standards for the Management of Open Fractures of the Lower Limb’ in 2009 principally recommended the measurement of life impact outcomes in the form of the Enneking Score,¹¹ which broadly assesses lower limb function in addition to the measurement of HRQoL, suggesting the Short-Study Form-36 (SF-36)⁹ or the Sickness Impact Profile (SIP)¹² as suitable tools. Despite these recommendations made over ten years ago, few studies have reported using these scores.

There was a high degree of variation in patient- and physician-reported outcome measurement instruments used to assess physical functioning. Of the 18 different outcome measurement instruments used, none were specific for measuring function following open lower limb fracture or traumatic injury. The majority were designed to measure the function of an anatomical area, generally in the context of elective orthopaedic surgery. HRQoL was measured 39 times across all studies with little consistency in the outcome measurement instruments used. A total of eight different HRQoL measures were identified.

This systematic review was ambitious in its data capture and included many relevant studies over ten years; it is unlikely that any potentially significant outcome term or outcome measurement instrument reported in the literature will not have been captured. Listing all outcome terms and definitions verbatim and summarizing source literature for each composite outcome measurement instrument identified in this study allows reviewers to trace outcomes identified in the COS back to their source. Due to the passage of time following the primary literature search, a search update was conducted. Undertaking staged systematic reviews is an accepted practice in COS development studies, allowing comparison of outcome lists against each other over time to identify if data saturation has been achieved.¹³ When comparing outcomes identified from the primary search (January 2009 to April 2017) and the search update (January 2017 to July 2019), no new standardized outcome headings were created. As such, it is unlikely that any additional new significant outcomes will have been reported since July 2019 to the point of publication of this study.

We limited the inclusion of studies to those published after 2009. This was done to ensure that the review remained focused on the relevant literature and manageable while avoiding unnecessary data extraction of potentially irrelevant literature; this approach is commensurate with COMET guidance.¹³ However, outcomes important to stakeholders may have been reported before 2009 and may have been missed. Although this is a possibility, it is unlikely, as clinically meaningful outcomes would likely have been reported over the last decade. The review was limited to studies published in the English language. However, this was mitigated by extracting outcomes from non-English-language articles reporting an English-language abstract.

The categorization of outcomes using the COMET Taxonomy of Outcomes is subjective, and as a result there may be conflict in the categorization of outcomes between the outcome terms and the outcome measurement instruments, leading to inconsistencies. While the COMET Taxonomy of Outcomes provides an excellent framework for categorizing reported outcomes, it was not designed to categorize outcome measurement instruments. However, categorizing outcomes and outcome measurement instruments was a method used to structure, organize, and present data to stakeholders.⁶ Using an outcome framework such as the COMET Taxonomy of Outcomes, it is possible to see the spread of outcomes across core areas and the underlying outcome domains, allowing COS developers to identify any apparent gaps in outcomes identified from the literature. Unexpected gaps, if present, may indicate that data saturation was not achieved. On interrogation of the outcomes spread across the COMET Taxonomy of Outcomes domains, there were no apparent areas where outcomes we may have expected to be reported were missing.

In conclusion, this systematic review highlights significant inadequacies in outcome reporting in the current body of literature on open lower limb fracture, both in terms of outcome heterogeneity and a failure to report on life impact outcomes measuring a patient’s functional recovery, despite guidance published in 2009 advocating the use of PROMs. There was an over-reliance on objective clinical outcomes reported in the literature. This systematic review supports the need to unify outcome collection and recognizes the need for higher-quality studies in this field. Developing a COS for open lower limb fracture will provide a consensus and guidance for collecting a homogeneous set of outcomes and outcome measures that will help address current limitations identified in this field.³