Abstract
Aims
Open lower limb fracture is life-changing, resulting in substantial morbidity and resource demand, while inconsistent outcome-reporting hampers systematic review and meta-analysis. A core outcome set establishes consensus among key stakeholders for the recommendation of a minimum set of outcomes. This study aims to define a core outcome set for adult open lower limb fracture.
Methods
Candidate outcomes were identified from a previously published systematic review and a secondary thematic analysis of 25 patient interviews exploring the lived experience of recovery from open lower limb fracture. Outcomes were categorized and sequentially refined using healthcare professional and patient structured discussion groups. Consensus methods included a multi-stakeholder two-round online Delphi survey and a consensus meeting attended by a purposive sample of stakeholders, facilitated discussion, and voting using a nominal group technique.
Results
Thematic analysis and systematic review identified 121 unique outcomes, reduced to 68 outcomes following structured discussion groups. Outcomes were presented to 136 participants who completed a two-round online Delphi survey. The Delphi survey resulted in 11 outcomes identified as consensus ‘in’ only. All outcomes were discussed at a consensus meeting attended by 15 patients, 14 healthcare professionals, 11 researchers, and one patient-carer. Consensus was achieved for a four-core outcome set: ‘Walking, gait and mobility’, ‘Being able to return to life roles’, ‘Pain or discomfort’, and ‘Quality of life’.
Conclusion
This study used robust consensus methods to establish a core outcome set that should be measured in all future research studies and audits of clinical practice without precluding the measurement of additional outcomes.
Cite this article: Bone Joint Res 2023;12(4):294–305.
Article focus
-
What outcomes are important to patients recovering from open lower limb fracture?
-
What constitutes the UK multi-stakeholder consensus for a core outcome set for adults recovering from open lower limb fractures?
Key messages
-
The core outcome set for open lower limb fractures is ‘Walking, gait and mobility’, ‘Being able to return to life roles’, ‘Pain or discomfort’, and ‘Quality of life’.
-
This constitutes the UK patient, healthcare professional, and researcher consensus on the minimum outcomes that should be measured on all skeletally mature patients recovering from open lower limb fracture in future clinical research and audits of clinical practice.
-
Further research is needed to establish how these outcomes should be measured, e.g. what standardized outcome definitions or measurement instruments should be used.
Strengths and limitations
-
This study involved the opinions and views of patients at each stage, used robust consensus methods to ensure that all stakeholder voices were heard, and was conducted in line with guidance from the Core Outcome Measures in Effectiveness Trials initiative.
-
Patients were sampled from the South and Midlands areas of England, and their views may not be representative of other global regions. It was unknown how many patients attending the consensus meeting had suffered a severe complication following open fracture, e.g. deep infection or nonunion.
Introduction
Open fracture is a life-altering injury affecting 30.7 per 100,000 adults each year.1 Open lower limb fracture causes substantial morbidity for patients and a large resource demand on trauma infrastructures.2,3 Treatment is complex, multifaceted, and subject to regional variation, and often results in considerable functional impairment,3,4 such that an optimal treatment pathway for these devastating injuries remains unclear. Inconsistency in outcome-reporting and poor-quality studies hamper attempts to evaluate new interventions to improve function, recovery experience, and investigation into the effects of regional variation of treatment on recovery.5
A core outcome set is an agreed, standardized set of outcomes to be measured and reported, as a minimum, in all trials for a specific population and health condition, and does not preclude the measurement of other outcomes in addition to the core outcome set.6 Core outcome set projects have been endorsed as a method to reduce outcome heterogeneity, outcome-reporting bias, and to promote better-quality research that is patient-centred through the involvement of key stakeholders.6-10 The development of a core outcome set for open lower limb fracture establishes a consensus among healthcare professionals, researchers, and patients over what outcomes are the most important to measure, with an overall aim of improving research and clinical practice in open lower limb fracture care. The core outcome set is anticipated for use in all trials of surgical, medical, and rehabilitation interventions in the setting of clinical research and routine clinical practice, e.g. local and national audit.
This study reports the consensus process undertaken involving key stakeholders to achieve a core outcome set, which defines what should be measured on skeletally mature patients recovering from open lower limb fracture.
Methods
Ethics and registration
The South-Central Research Ethics Committee (REC) granted ethical approval for this study on 1 March 2018 (REC reference: 18/SC/0051, IRAS project ID: 235150). The Core Outcomes for Open Lower Limb Fracture (CO-OLLF) study was prospectively registered on the Core Outcome Measures in Effectiveness Trials (COMET) database,11 and was adopted by the National Institute for Health and Care Research (NIHR) Clinical Research Network (CRN) (Protocol number: 13257).
Design
Candidate outcomes were identified from a systematic literature review12 and a secondary qualitative thematic analysis of in-depth patient interviews that explored the lived experience of recovery from open lower limb fracture. Candidate outcomes were fed back to stakeholders in structured discussion groups and a two-round Delphi survey before a final review at a consensus meeting. This study was designed and reported in accordance with the COMET Initiative guidance7,9 and the Core Outcome Set-STAndards for Reporting statement,10 respectively.
Population
The population and health condition used for the CO-OLLF study was defined as all skeletally mature patients (age greater than 18 years) following open lower limb fracture of any grade distal to the acetabulum.12 Health economic outcomes were considered out of scope.13
Information sources
Outcomes important to healthcare professionals and researchers were identified using a systematic literature review to collect, summarize, and categorize contemporary outcomes reported in clinical studies on skeletally mature patients following open lower limb fracture over ten years from 2009.5
Outcomes important to patients were identified through a secondary thematic analysis of 25 in-depth interview transcripts that explored the lived experience of recovery following open lower limb fracture. Interview participants were recruited as part of the WOLLF HTA-10/57/20 trial.4 A purposive sampling strategy was used to ensure a range of sex, age, mechanism of injury, the severity of open lower limb fracture, and time since injury. Interviews were conducted between two and four years post-injury to provide an overview of the recovery experience. The primary data collection, qualitative analysis, results, and further methodological detail regarding the research team, reflexivity, study design, participant demographics, and interview structure are published elsewhere.3,4
A secondary thematic analysis of interview transcripts was undertaken. Open coding was used to identify and organize text extracts representing outcome codes. Outcome codes were interpreted to create outcome themes. Outcome themes were then grouped and organized under appropriate outcome domains using the COMET taxonomy of outcomes as a framework.14 Data were organized using NVivo V.12 (QSR International, UK). To facilitate reflection on the process of interpretation, a sample of five interview transcripts were coded in duplicate by ALA and ET before meeting to discuss and compare identified codes, themes, and interpretations. Rigour was demonstrated through trustworthiness.15 Outcome themes grouped under outcome in the COMET taxonomy outcome domains were presented to patients and healthcare professionals at structured discussion groups, a Delphi survey, and at a consensus meeting in an iterative process to enhance clarity. See Supplementary Methods for justification and a detailed summary of the thematic analysis methodology.
Outcomes identified from the systematic literature review and the qualitative analysis were combined and rationalized before being categorized and organized using the COMET taxonomy of outcomes to create an inventory of outcomes for open lower limb fracture. Outcomes were assigned a number to aid later discussion and tracking through the consensus process. The COMET taxonomy of outcomes is a categorization system that provides sufficient granularity and scope for classifying all medical outcomes in core outcome set development.14 The inventory of outcomes was presented to a purposive sample of healthcare professionals and researchers, followed by patients in two structured discussion groups. Structured discussion groups were used to refine and review outcomes for duplication, use of language, and understandability before the consensus process. The resulting list of outcomes was reviewed and refined by the study management group (consisting: ALA, ET, RF, MLC, and XLG) before being taken forward to the Delphi survey.
Delphi survey
A two-round Delphi survey was conducted using COMET DelphiManager (University of Liverpool, UK)16 between 9 February 2019 and 19 March 2019. Participants were recruited from three key stakeholder groups: patients, healthcare professionals, and researchers (clinical academics and people working in clinical musculoskeletal research, e.g. members of clinical trial teams). Study adoption by the NIHR CRN facilitated multisite recruitment. Patients were recruited from five UK major trauma centres (MTCs). Patients were eligible if they were over the age of 18 years and had sustained an open lower limb fracture (as defined above) at any point before participation. Healthcare professionals and researchers were recruited through cascade emails sent through the Major Trauma Network, the Orthopaedic Trauma Society, and the British Association of Plastic and Reconstructive Surgeons. Healthcare professionals and researchers were eligible for recruitment if their professional role involved the regular clinical care of patients following open lower limb fracture or if they actively participated in orthopaedic trauma research, respectively.
In each survey round, participants were presented with the inventory of outcomes for open lower limb fracture and asked to score the outcomes for importance using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) 1 to 9 Likert scale.17,18 The scale was displayed as 1 to 3 labelled ‘not important’, 4 to 6 labelled ‘important but not critical’, and 7 to 9 labelled ‘critical’. Participants could suggest additional outcomes at the end of round 1. The study management group reviewed proposed outcomes, and outcomes identified as novel were added to round 2. In round 2, participants were shown a bar chart of round 1 scores for each outcome by stakeholder group, overall score, and their first-round score. They were then asked to reflect on their scoring, considering the additional information before re-scoring each outcome. The consensus criteria used to identify outcomes as consensus-in, no-consensus, and consensus-out were those recommended by COMET and defined a priori (Table I).9 No outcomes were dropped between rounds.
Table I.
Criteria | Description |
---|---|
‘Consensus-in’ | Over 70% of participants in each stakeholder group score the outcome ‘critical’ (7 to 9) or ‘in’, AND less than 15% of participants score outcome ‘not important’ (1 to 3) or ‘out’. |
‘Consensus-out’ | Over 70% of participants in each stakeholder group score the outcome ‘not important’ (1 to 3) or ‘out’, AND less than 15% of participants score outcome ‘critical’ (7 to 9) or ‘in’. |
‘No-consensus’ | Anything else. |
Consensus meeting
The results of the Delphi survey were presented at a one-day face-to-face consensus meeting hosted at a purpose-built events centre on 26 March 2019. Patients, healthcare professionals, and researchers who had completed both rounds of the Delphi survey were invited to attend. Participants were sampled purposively using a sampling matrix to ensure a balanced representation of patients in terms of age, sex, injury severity, and time since injury, also including healthcare professionals and researchers to achieve a mixed sample of professional backgrounds. All participants were sent a detailed pre-meeting reading pack summarizing the project, results of the Delphi survey, and meeting agenda. The meeting structure and methods were guided by the COMET Handbook9 and adapted from the James Lind Alliance Priority Setting Partnerships methods for a final priority setting meeting.18
The meeting consisted of three separate sessions, and participants were split into three groups for sessions one and two, which were independently facilitated by members of the study management group (see Figure 1 for a schematic representation of the meeting structure). Initially, each group selected their top two or three outcomes from the ‘no-consensus’ outcomes identified in the Delphi survey for inclusion in the second session. Then, each group ranked a top ten list of outcomes from all ‘consensus-in’ outcomes identified in the Delphi survey second round plus outcomes selected for inclusion earlier. Finally, groups came together, and anonymous electronic voting was undertaken using TurningPoint software and handsets (Turning Technologies, USA). After removing duplicates, all outcomes from the groups’ top 10's were added to a Microsoft PowerPoint (Microsoft Corporation, USA) presentation template to facilitate electronic voting for inclusion in the final core outcome set. Electronic voting enabled the presentation of voting results on each outcome immediately to aid discussion. Participants were given the scoring options of 1 to 3: ‘not important’, ‘important but not critical’, and ‘critical’. Outcomes were voted on in two rounds with the facilitation of discussion on each outcome before repeated voting. Outcomes reaching ‘consensus-in’ were included in the core outcome set; all others were excluded. The core outcome set was then presented to the participants and ratified.
Fig. 1
The meeting was chaired by a facilitator with experience in running consensus meetings (MLC). Group facilitators ensured that contrasting views were actively sought, and all participants had equal opportunity to contribute to the discussion throughout the day. Each participant was encouraged to share their opinions on an outcome before any voting or ranking exercise was undertaken. A modified nominal group technique (NGT) was used to facilitate consensus. NGT is a well-established consensus method used to make decisions efficiently while taking everyone’s views into account.9,19 NGT eliminates duplicate ideas and allows all individual opinions to be considered, which influences the group intending to converge opinions to achieve a consensus.9,19
Other analyses and statistical considerations
We analyzed participant attrition rate and outcome scoring changes between Delphi survey rounds. Medians and standard deviations were calculated for the score (1 to 3) of each outcome scored in the consensus meeting electronic-voting session using TurningPoint software.
There is currently no recommended minimum number of participants to include in a Delphi survey and no requirement to demonstrate a statistically representative sample. In previous studies, between ten and 15 participants per stakeholder group have yielded satisfactory results.9,19 NGT does not depend on statistical power, and there is no robust or commonly used method for calculating the required number of participants.9
Results
Information gathering
A systematic review describing outcomes reported in the literature has been previously published;5 1,803 outcomes were identified from 786 studies and consolidated to 82 standardized outcome headings (Supplementary Figure a).
Thematic analysis of 25 interview transcripts identified 1,121 codes in the data representing an explicit or underlying outcome that may be important to measure in patients recovering from open lower limb fracture (see Supplementary Methods for examples of patient quotations representative of underlying outcome codes). Interpreting the outcome codes resulted in the development of 88 outcome themes and sub-themes (see the coding tree detailed in Supplementary Figure b). Duplicate data coding by ALA and ET demonstrated consistency over the identification of outcome codes in the data.
Outcome headings and themes from the systematic review and qualitative analysis, respectively, were brought together to create the inventory of outcomes for open lower limb fracture. The resulting inventory of outcomes consisted of a long list of 150 outcomes categorized using the COMET taxonomy of outcomes to 18 outcome domains within five core areas. Following rationalization of outcomes byALA, structured discussion groups, and a study management group meeting, 68 outcomes were shortlisted for the Delphi survey (see Figure 2 for a schematic diagram of outcome rationalization, with further detail provided in Supplementary Table i).
Fig. 2
Delphi survey
A total of 136 participants completed both rounds of the Delphi survey, of whom 55 (40%), 73 (54%), and eight (6%) were patients, healthcare professionals, and researchers, respectively (see Table II for participant characteristics). Six and 11 outcomes achieved ‘consensus-in’ in the first and second rounds, respectively. No outcomes met the ‘consensus-out’ criteria. The remaining 59 outcomes were ‘no-consensus’ after the second round (see Supplementary Table ii for a complete list of outcomes and their scoring for ‘consensus-in’). Participants in the first round suggested 13 additional outcomes. The study management group rejected 11, and two were accepted for inclusion in the subsequent Delphi survey (see Supplementary Table iii for accepting or rejecting justification). Figure 3 shows a schematic flow diagram of the Delphi survey and consensus meeting results. The between round attrition rate was 27%, with attrition spread relatively evenly across the three stakeholder groups (26% patients, 28% healthcare professionals, and 27% for researchers).
Fig. 3
Table II.
Demographic | Delphi survey | COS consensus meeting | |
---|---|---|---|
Round 1 n = 187 |
Round 2 n = 136 |
n = 41 | |
Stakeholder group, n (%) | |||
Patients | 74 (40) | 55 (40) | 15 (37) |
Anatomical area of open fracture* | |||
Open fracture of the thigh (femur) | 1 (1) | 1 (1) | N/A |
Open fracture of the leg including ankle (tibia/fibula) | 61 (85) | 43 (81) | 14 (93) |
Open fracture of the foot (all bones of the foot including phalanges) | 5 (7) | 5 (9) | 1 (7) |
Open fracture of the thigh and leg | 1 (1) | 1 (1) | N/A |
Open fracture of the leg and foot | 3 (5) | 2 (4) | N/A |
Open fracture of the thigh, leg, and foot | 1 (1) | 1 (1) | N/A |
Indication of severity of open fracture† | |||
Open fracture closed primarily | 16 (22) | 11 (20) | 6 (40) |
Open fracture required a skin graft or muscle flap | 52 (72) | 40 (74) | 9 (60) |
Open fracture resulting in an amputation | 4 (6) | 3 (6) | N/A |
Patient carers | N/A | N/A | 1 (2) |
Healthcare professionals‡ | 102 (55) | 73 (54) | 14 (34) |
Anaesthetist or intensive care physician | 1 (1) | 1 (2) | N/A |
General practitioner | 3 (3) | 3 (4) | N/A |
Nurse (including advanced nurse practitioners) | 9 (8) | 7 (10) | 3 (21) |
Physician (e.g. orthogeriatrician) | 3 (3) | 3 (4) | 1 (7) |
Physiotherapist | 11 (10) | 8 (11) | 1 (7) |
Plastic and reconstructive surgeon | 23 (21) | 14 (20) | 3 (21) |
Psychologist (e.g. trauma psychologist) | 1 (1) | 1 (2) | 1 (7) |
Trauma and orthopaedic surgeon | 55 (50) | 35 (45) | 3 (21) |
Other | 3 (3) | 0 | 2 (16) |
Researchers§ | 11 (5) | 8 (6) | 11 (27) |
Post-doctoral researcher | 3 (11) | 1 (14) | 2 (18) |
Professor or associate professor | 4 (14) | 0 | 5 (46) |
Research nurse/associate | 6 (21) | 3 (43) | 2 (18) |
Student (MSc or PhD) | 8 (29) | 0 | 1 (9) |
Trial manager | 2 (7) | 2 (29) | 1 (9) |
Other | 5 (18) | 1 (14) | N/A |
Sex, n (%) | |||
Male | 123 (66) | 85 (62) | 23 (57) |
Female | 64 (34) | 51 (38) | 17 (43) |
Patients: Male | 43 (58) | 31 (56) | 9 (60) |
Female | 31 (42) | 24 (44) | 6 (40) |
Age (years), n (%) | |||
Patients: 20 to 29 | 13 (17) | 9 (16) | 1 (6) |
30 to 39 | 10 (14) | 8 (15) | 3 (20) |
40 to 49 | 11 (15) | 7 (13) | 1 (6) |
50 to 59 | 22 (30) | 17 (31) | 6 (40) |
60 to 69 | 6 (8) | 3 (5) | 1 (6) |
over 70 | 12 (16) | 11 (20) | 3 (20) |
Mean, yrs | 49 (20 to 82) | 51 (20 to 81) | 53 (26 to 76) |
Mean male, yrs | 44 (21 to 76) | 44 (21 to 76) | 48 (26 to 72) |
Mean female, yrs | 58 (20 to 82) | 59 (20 to 82) | 60 (31 to 76) |
Professionals: 20 to 29 (healthcare professionals and researchers)¶ |
8 (7) | 8 (10) | N/A |
30 to 39 | 51 (46) | 36 (45) | N/A |
40 to 49 | 36 (32) | 23 (29) | N/A |
50 to 59 | 13 (11) | 10 (13) | N/A |
60 to 69 | 3 (3) | 2 (2) | N/A |
over 70 | 1 (1) | 1 (1) | N/A |
Mean, yrs | 41 | 41 | N/A |
Patients sustained open lower limb fracture on military duty, n (%) | 3 (4) | 2 (4) | N/A |
Professionals employed by the military, n (%) | 9 (9) | 6 (8) | N/A |
Plastic and reconstructive surgeon | 6 (66.7) | 5 (83.3) | N/A |
Trauma and orthopaedic surgeon | 3 (33.3) | 1 (16.7) | N/A |
-
*
Two patients did not state the anatomical area of open fracture.
-
†
Two patients did not indicate the severity of their open fracture.
-
‡
Seven healthcare professionals self-identified as researchers when selecting stakeholder group.
-
§
A total of 17 researchers self-identified as healthcare professionals when selecting stakeholder group.
-
¶
One healthcare professional entered an incorrect date of birth.
-
COS, core outcome set; N/A, not applicable.
Consensus meeting
A total of 15 patients, 14 healthcare professionals, 11 researchers, and one patient carer attended the consensus meeting (see Table II for participant characteristics). The majority of patients were male (nine (60%)), having sustained an open lower limb fracture of the tibia/fibula (14 (93%)) that required a skin graft or muscle flap (nine (60%)), and the mean age of patients was 53 years (26 to 76). See Table III for a complete list of outcomes by outcome number discussed at the consensus meeting, and Figure 3 for a schematic of results. The first session led to creating two new outcomes and the selection of ten outcomes out of the 59 ‘no-consensus’ outcomes for later consideration (Figure 3). The groups’ top ten outcomes included 18 outcomes after removing duplicates. In the workshop’s final session, walking, gait, and mobility, being able to return to life roles (e.g. caring, work, and military duty), quality of life, and pain or discomfort outcomes were agreed upon (see Table IV for electronic voting results in full).
Table III.
COMET Taxonomy of Outcome Domains | Outcome |
---|---|
Physical functioning | 1. Ability to do activities of daily living |
2. Ability to kneel | |
3. Ability to balance | |
4. Ability to be independent | |
5. Ability to drive | |
6. Walking, gait and mobility | |
7. Ability to be able to dress and do self and personal care | |
8. Ability to have intimate relationships | |
9. Ability to exercise | |
10. Ability to comfortably wear shoes and clothes of your choice | |
11. Lower limb function | |
12. Range of motion | |
Social functioning | 13. Ability to participate in social events |
14. Ability to start, maintain and develop relationships | |
Role functioning | 15. Being able to care for other people and/or animals |
16. Being able to return to life roles (e.g. caring, work, military duty) | |
Emotional functioning and wellbeing | 17. Appearance of lower limb e.g. scars, flaps and swelling |
18. Change in body shape or weight | |
19. Feelings of self-identity and body wholeness | |
20. Feelings of disablement, being damaged or broken | |
21. Feelings of giving up and hopelessness | |
22. Feeling of loss | |
23. Feelings of frustration and anger | |
24. Feelings of hopefulness, positivity and determination | |
25. Isolation, loneliness and lack of emotional support | |
26. Low mood | |
27. Ability to move on and feeling of getting life back | |
28. Having clear expectations and views on the future with lack of uncertainty | |
29. Tolerance of uncertainty and living in fear e.g. fear of falling or going outside and fear for the future | |
30. Being anxious or stressed | |
31. Loss of confidence or control and feeling of limitation | |
32. Experience of flashbacks to the traumatic event | |
33. Emotional vulnerability, volatility and fragility. | |
34. Ability to accept the event and life change | |
35. Feelings of struggle or fighting against the body | |
36. Maintaining a feeling of dignity | |
Cognitive functioning | 37. Lack of concentration and focus |
Blood and lymphatic system outcomes | 38. Venous thromboembolism (blood clot in legs or lungs) |
General outcomes | 39. Sleep and fatigue |
40. Number of unplanned hospital re-admissions | |
41. Length of hospital stay | |
42. Number of outpatient appointments | |
43. Number of Primary Care visits | |
Musculoskeletal and connective tissue outcomes | 44. Metalwork failure |
45. Complications resulting from soft tissue (muscle flaps or grafts) | |
46. Amputation | |
47. Bone healing | |
48. Malunion, alignment and rotational deformity | |
49. Deep infection | |
50. Surface infection | |
51. Leg or bone shortening | |
52. Change in sensation | |
53. Muscle weakness | |
54. Pain or discomfort | |
55. Unplanned return to the operating theatre | |
Global quality of life | 56. Quality of life |
Perceived health status | 57. Degree of perceived loss of general health and wellbeing |
Delivery of care | 58. Quality of communication of healthcare professionals to patients |
59. Satisfaction with care | |
60. Tolerability and acceptability of surgical intervention e.g. external fixator | |
Personal circumstances | 61. Change in lifestyle |
62. Being compensated financially for injury | |
63. Personal cost of treatment and care following injury | |
64. Effect of injury on personal finances | |
Adverse events and effects | 65. Number of adverse events |
66. Side effects of medication | |
67. Poor experience during anaesthesia | |
Mortality | 68. Survival |
Musculoskeletal and connective tissue outcomes | 69. Chronic pain (Added following Delphi survey round 1) |
General outcomes | 70. Length of stay (Super Spell) (Added following Delphi survey round 1) |
71. Number and length of health encounters (Added at consensus meeting session 1: outcomes - 42, 43 and 70 rationalized to create outcome 71) | |
Musculoskeletal and connective tissue outcomes | 72. Injury characteristics and surgical outcomes (Added at consensus meeting session 1: outcomes - 44, 45, 48 and 55 to create outcome: 72) |
-
COMET, Core Outcome Measures in Effectiveness Trials.
Table IV.
Outcome | Number of electronic votes counted in session 3, round 1, n = 41 (%) | Number of electronic votes counted in session 3, round 2, n = 41 (%) | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
1. Not important | 2. Important but not critical | 3. Critical | Median | SD | 1. Not important | 2. Important but not critical | 3. Critical | Median | SD | |
1. Ability to do activities of daily living | 6 (15) | 11 (28) | 22 (56) | 3 | 0.74 | 10 (26) | 3 (8) | 26 (67) | 3 | 0.87 |
4. Ability to be independent | 6 (15) | 8 (21) | 25 (64) | 3 | 0.75 | 19 (51) | 2 (5) | 16 (43) | 1 | 0.97 |
6. Walking, gait, and mobility* | 0 (0) | 8 (21) | 30 (79) | 3 | 0.41 | N/A | N/A | N/A | N/A | N/A |
16. Being able to return to life roles (e.g. caring, work, and military duty)* | 3 (8) | 3 (8) | 33 (85) | 3 | 0.58 | N/A | N/A | N/A | N/A | N/A |
19. Feelings of self-identity and body wholeness | 23 (57) | 16 (40) | 1 (2) | 1 | 0.55 | 34 (92) | 3 (8) | 0 (0) | 1 | 0.27 |
21. Feelings of giving up and hopelessness | 14 (35) | 14 (35) | 12 (30) | 2 | 0.8 | 23 (61) | 1 (3) | 14 (37) | 1 | 0.96 |
24. Feelings of hopefulness, positivity, and determination | 23 (59) | 10 (26) | 6 (15) | 1 | 0.74 | 25 (68) | 3 (8) | 9 (24) | 1 | 0.86 |
27. Ability to move on and feeling of getting life back | 15 (38) | 7 (18) | 17 (44) | 2 | 0.9 | 22 (61) | 0 (0) | 14 (39) | 1 | 0.97 |
40. Number of unplanned hospital readmissions | 20 (53) | 10 (26) | 8 (21) | 1 | 0.8 | 25 (64) | 4 (10) | 10 (26) | 1 | 0.87 |
46. Amputation | 15 (38) | 9 (22) | 16 (40) | 2 | 0.88 | 20 (54) | 3 (8) | 14 (38) | 1 | 0.94 |
47. Bone healing | 11 (28) | 13 (32) | 16 (40) | 2 | 0.81 | 24 (60) | 1 (2) | 15 (38) | 1 | 0.96 |
49. Deep infection | 10 (26) | 11 (28) | 18 (46) | 2 | 0.82 | 16 (40) | 1 (2) | 23 (57) | 3 | 0.97 |
54. Pain or discomfort† | 5 (12) | 12 (30) | 23 (57) | 3 | 0.71 | 6 (15) | 3 (8) | 30 (77) | 3 | 0.74 |
56. Quality of life* | 1 (2) | 7 (18) | 32 (80) | 3 | 0.47 | N/A | N/A | N/A | N/A | N/A |
57. Degree of perceived loss of general health and wellbeing | 23 (59) | 13 (33) | 3 (8) | 1 | 0.64 | 27 (71) | 1 (3) | 10 (26) | 1 | 0.88 |
58. Quality of communication of healthcare professionals to patients | 16 (40) | 14 (35) | 10 (25) | 2 | 0.79 | 21 (58) | 1 (3) | 14 (39) | 1 | 0.97 |
68. Survival | 21 (54) | 6 (15) | 12 (31) | 1 | 0.89 | 21 (54) | 6 (15) | 12 (31) | 1 | 0.89 |
72. Injury characteristics and surgical outcome | 15 (38) | 6 (15) | 18 (46) | 2 | 0.92 | 10 (27) | 1 (3) | 26 (70) | 3 | 0.89 |
-
*
Outcomes reaching ‘consensus-in’ at round 1.
-
†
Outcomes reaching ‘consensus-in’ at round 2.
-
N/A, not applicable; SD, standard deviation.
Electronic voting in this session stimulated debate over whether the ‘bone healing’ and ‘deep infection’ outcomes should be included in the core outcome set. The patients felt that fundamentally, being able to walk, return to work, and maintain a good quality of life are the most important outcomes to achieve in recovery and that bone healing and infection were secondary. The inclusion of ‘deep infection’ was a sticking point for several consultant surgeons, arguing that minimizing the risk of infection is a guiding principle of modern practice and is fundamentally an essential component of achieving a positive treatment outcome. A healthcare professional raised a counterargument, pointing out that the core outcome set is intended for use in all research and clinical practice on patients following open lower limb fracture. Studies on rehabilitation or psychosocial support interventions would not consider infection a key outcome to measure. An additional point supported this argument that in any surgical intervention study, infection would be regarded as an adverse event; as such, its measurement would be mandatory. The group agreed that objective surgical outcomes, including ‘deep infection’ and ‘bone healing’, are important and need to be measured in surgical intervention trials but not across all open lower limb fracture studies. Therefore, they should not be included in the core outcome set.
The four ‘consensus-in’ outcomes were presented to the group and ratified as the core outcome set for open lower limb fracture (see Figure 3).
Discussion
This study provides the first combined patient, healthcare professional, and researcher consensus on what outcomes to measure as part of routine clinical practice and research on patients following open lower limb fracture. Our systematic review identifies that the existing literature on open lower limb fracture is hampered by outcome heterogeneity and a lack of evidence to determine which outcomes are important to patients when recovering from open lower limb fracture.5 This study addresses these shortcomings by using consensus methods that ensure equal representation of stakeholder views,9 to identify outcomes that have been scored as critically important by more than 70% of patients, healthcare professionals, and researchers.
This study meets the COMET Core Outcome Set-STAndards for Development (COS-STAD),7 an internationally agreed set of standards recommended to ensure methodologically robust core outcome set development projects.
The Delphi survey round attrition rate was 27%, higher than the average attrition rate of 12%, calculated from 19 core outcome set studies registered on the COMET database in 2018 reporting a Delphi survey.19 The degree of non-response after the first Delphi round has been demonstrated to be highly variable between studies and can depend on the timing of the Delphi rounds, survey length, the time elapsed between the first and second rounds, and the recruitment method of participants. Several strategies were used to reduce attrition between rounds, including multiple reminder emails and patient follow-up by telephone. It may have been possible to minimize between-round attrition by decreasing the length of the survey. However, this needed to be balanced with presenting a comprehensive long list of potential core outcomes.
The patient sample recruited for the Delphi survey and consensus meeting was representative of age and sex for patients presenting to English MTCs.20 Patients were recruited from MTCs in Oxford, Bristol, and Coventry; thus, the patient sample is likely to have been mixed in terms of social deprivation due to sociodemographic variation within and between these regions. However, the patient sample was predominantly from the South and Midlands areas of England and may not be representative of worldwide patient views and opinions. We cannot comment on how representative the Delphi survey results are of the race and ethnic diversity of the UK population as these data were not collected. While interview transcripts and patients included in the Delphi survey were representative of experiences of severe complications during their treatment, such as a deep infection or nonunion requiring a return to theatre, it was unknown how many patients attending the consensus meeting had undergone such an experience. Poor representation of patients who underwent treatment complications may explain the exclusion of deep infection and bone union in the core outcome set. Without direct experience of these complications, it is unlikely that patients will have insight into the seriousness of such adverse events and, therefore, not advocate for the inclusion of related outcomes. However, this issue was discussed during the final voting session of the consensus meeting, where surgeons explained the impact of suffering such a complication. Patients felt that the negative life impact of such an event would be sufficiently captured by the ‘walking, gait, and mobility’ and ‘quality of life’ outcomes. It was also unexpected that no outcomes directly representative of the emotional functioning and wellbeing outcome domain were included in the core outcome set. This may have resulted from excluding an overarching candidate core outcome for this domain, e.g. ‘stress and anxiety’. However, three emotional functioning and wellbeing outcomes (outcomes 19, 21, and 24; Table III) did reach the final voting session, but all received a majority vote of ‘not important’ in both voting rounds. Participants felt that emotional functioning and wellbeing would be sufficiently measured by ‘quality of life’ during the discussion.
In conclusion, this study identified a four-outcome core outcome set (Figure 3) representing the recommended outcomes to measure as a minimum in future trials, systematic review, clinical research, and audit on adult open lower limb fracture. Further work is required to identify how these outcomes should be measured.
It is hoped that consistent outcome reporting guided by the CO-OLLF core outcome set will guarantee that future research addresses outcomes agreed upon as important by all stakeholders, facilitates evidence synthesis across studies, and ultimately leads to enhanced patient care and improvement of treatment results following open lower limb fracture.
References
1. Court-Brown CM , Bugler KE , Clement ND , Duckworth AD , McQueen MM . The epidemiology of open fractures in adults. A 15-year review . Injury . 2012 ; 43 ( 6 ): 891 – 897 . Crossref PubMed Google Scholar
2. Costa ML , Achten J , Bruce J , et al. Effect of negative pressure wound therapy vs standard wound management on 12-month disability among adults with severe open fracture of the lower limb: The WOLLF randomized clinical trial . JAMA . 2018 ; 319 ( 22 ): 2280 – 2288 . Crossref PubMed Google Scholar
3. Rees S , Tutton E , Achten J , Bruce J , Costa ML . Patient experience of long-term recovery after open fracture of the lower limb: a qualitative study using interviews in a community setting . BMJ Open . 2019 ; 9 ( 10 ): e031261 . Crossref PubMed Google Scholar
4. Costa ML , Achten J , Bruce J , et al. Negative-pressure wound therapy versus standard dressings for adults with an open lower limb fracture: the WOLLF RCT . Health Technol Assess . 2018 ; 22 ( 73 ): 1 – 162 . Crossref PubMed Google Scholar
5. Aquilina AL , Claireaux H , Aquilina CO , et al. What outcomes have been reported on patients following open lower limb fracture, and how have they been measured? Bone Joint Res . 2023 ; 12 ( 2 ): 138 – 146 . Crossref PubMed Google Scholar
6. Williamson PR , Altman DG , Blazeby JM , et al. Developing core outcome sets for clinical trials: issues to consider . Trials . 2012 ; 13 ( 1 ): 132 . Crossref PubMed Google Scholar
7. Kirkham JJ , Davis K , Altman DG , et al. Core Outcome Set-STAndards for Development: The COS-STAD recommendations . PLoS Med . 2017 ; 14 ( 11 ): e1002447 . Crossref PubMed Google Scholar
8. Ollivere BJ , Marson BA , Haddad FS . Getting the right answer: core outcome sets in orthopaedics . Bone Joint J . 2019 ; 101-B ( 3 ): 233 – 235 . Crossref PubMed Google Scholar
9. Williamson PR , Altman DG , Bagley H , et al. The COMET Handbook: version 1.0 . Trials . 2017 ; 18 ( Suppl 3 ): 280 . Crossref PubMed Google Scholar
10. Kirkham JJ , Gorst S , Altman DG , et al. Core Outcome Set-STAndards for Reporting: The COS-STAR Statement . PLoS Med . 2016 ; 13 ( 10 ): e1002148 . Crossref PubMed Google Scholar
11. No authors listed . Core Outcomes for research on Open Lower Limb Fractures (CO-OLLF): Development of a core outcome set and core outcome measurement set . COMET Initiative . https://www.comet-initiative.org/Studies/Details/688 ( date last accessed 5 April 2023 ). Google Scholar
12. Trompeter AJ , Knight R , Parsons N , Costa ML . The Orthopaedic Trauma Society classification of open fractures . Bone Joint J . 2020 ; 102-B ( 11 ): 1469 – 1474 . Crossref PubMed Google Scholar
13. Png ME , Petrou S , Bourget-Murray J , Knight R , Trompeter AJ , Costa ML . Association between the Orthopaedic Trauma Society classification of open fractures and economic costs . Bone Joint J . 2022 ; 104-B ( 3 ): 408 – 412 . Crossref PubMed Google Scholar
14. Dodd S , Clarke M , Becker L , Mavergames C , Fish R , Williamson PR . A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery . J Clin Epidemiol . 2018 ; 96 ( 5 ): 84 – 92 . Crossref PubMed Google Scholar
15. Shenton AK . Strategies for ensuring trustworthiness in qualitative research projects . EFI . 2004 ; 22 ( 2 ): 63 – 75 . Crossref Google Scholar
16. No authors listed . DelphiManager . COMET Initiative . 2023 . www.comet-initiative.org/delphimanager/ ( date last accessed 9 March 2023 ). Google Scholar
17. Guyatt GH , Oxman AD , Kunz R , et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes . J Clin Epidemiol . 2011 ; 64 ( 4 ): 395 – 400 . Crossref PubMed Google Scholar
18. No authors listed . The James Lind Alliance . National Institute for Health and Care Research . 2023 . www.jla.nihr.ac.uk ( date last accessed 9 March 2023 ). Google Scholar
19. Gargon E , Gorst SL , Williamson PR . Choosing important health outcomes for comparative effectiveness research: 5th annual update to a systematic review of core outcome sets for research . PLoS ONE . 2019 ; 14 ( 12 ): e0225980 . Crossref PubMed Google Scholar
20. Young K , Aquilina A , Chesser TJS , et al. Open tibial fractures in major trauma centres: A national prospective cohort study of current practice . Injury . 2019 ; 50 ( 2 ): 497 – 502 . Crossref PubMed Google Scholar
Author contributions
A. L. Aquilina: Conceptualization, Formal analysis, Project administration, Writing – original draft.
H. Claireaux: Data curation, Formal analysis, Writing – original draft.
C. O. Aquilina: Data curation.
E. Tutton: Conceptualization, Data curation, Formal analysis, Methodology, Writing – review & editing.
R. Fitzpatrick: Conceptualization, Data curation, Formal analysis, Methodology, Writing – review & editing.
M. L. Costa: Supervision, Writing – review & editing.
X. L. Griffin: Conceptualization, Funding acquisition, Supervision, Writing – review & editing.
Funding statement
The authors disclose receipt of the following financial or material support for the research, authorship, and/or publication of this article: funding from the National Institute for Health and Care Research Clinician Scientist Award CS-2017-17-020.
ICMJE COI statement
X. L. Griffin reports an institutional grant (paid to University of Oxford) from the National Institute for Health and Care Research (NIHR) Health Technology Assessment Programme, related to this study, as well as multiple other grants from the NIHR which were not related to the study.
Acknowledgements
CO-OLLF Delphi survey and consensus meeting collaborators (patients, healthcare professionals, and researchers). A special thank you to Elizabeth Baird for her role as the study patient and public involvement partner and her participation in all patient-facing activities ensuring the patient's voice was heard. Thank you to Lauren Exell for all her administrative help throughout the study. This study was funded by an AO UK Major Research grant (WE/VMC 24 October 2017) and was supported by the National Institute for Health Research and Oxford Biomedical Research Centre. We are grateful for the support of the England and Wales Major Trauma Network, The Trauma, Audit and Research Network, the Orthopaedic Trauma Society, and the British Association of Plastic Reconstructive and Aesthetic Surgeons.
Ethical review statement
The South-Central Research Ethics Committee (REC) granted ethical approval for this study on 1 March 2018 (REC reference: 18/SC/0051, IRAS project ID: 235150). The CO-OLLF study was prospectively registered on the COMET database (http://www.comet-initiative.org/Studies/Details/688) and was adopted by the National Institute of Health Research Clinical Research Network (Protocol number: 13257).
Open access funding
The authors report that they received open access funding for their manuscript from the National Institute for Health and Care Research CS-2017-17-020.
Follow A. Leonard. Aquilina @alexaquilina87
Follow H. Claireaux @harryclax
Follow X. L. Griffin @xlgriffin
Follow the Blizard Institute, Barts and the London School of Medicine and Dentistry @QMUL
Supplementary material
Text describing a detailed secondary thematic analysis methodology. Figures showing coding trees presenting standardized outcome headings identified from the literature, and outcome themes interpreted from the thematic analysis categorized using the Core Outcomes Measures in Effectiveness Trials Initiative, Taxonomy of Outcomes. Tables illustrating the progression of outcomes from the inventory of outcomes to the Delphi survey, and additional outcomes suggested at the Delphi survey and justification for inclusion or exclusion.
© 2023 Author(s) et al. This is an open-access article distributed under the terms of the Creative Commons Attribution Non-Commercial No Derivatives (CC BY-NC-ND 4.0) licence, which permits the copying and redistribution of the work only, and provided the original author and source are credited. See https://creativecommons.org/licenses/by-nc-nd/4.0/