Impact of the PROFHER trial findings on surgeons’ clinical practice

Participants

In September 2016, surgeons were invited to participate in an online questionnaire via membership emails through the British Orthopaedic Association (BOA) and British Elbow and Shoulder Society (BESS). The email asked members to complete the questionnaire once, and only if they “currently treat patients for a proximal humerus fracture”. A reminder email was sent to potential participants after two weeks. In order to improve response rates, participants were offered the opportunity to enter into a prize draw to win a £100 gift voucher upon questionnaire completion. A winner was chosen at random two weeks following closure of the online questionnaire.

Ethical approval was gained from the University of York Department of Health Sciences Research Governance Committee and permissions from BOA and BESS research committees before disseminating the survey. Surgeons were not obliged to take part in this study and were given the opportunity to withdraw at any time during completion of the online questionnaire.

Questionnaire

Questionnaire items were generated through discussion between authors, which included expert researchers and a consultant orthopaedic surgeon who worked together on the PROFHER trial. The questionnaire draft was then piloted with a sample of six orthopaedic surgeons (members of BESS) in order to test the feasibility, ease of completion and face validity of questionnaire items prior to wider dissemination. A ‘think aloud’ process was undertaken,⁸ whereby each surgeon was asked to complete the questionnaire and talk through their responses with a researcher (LK) as they completed each item. By surgeons voicing their thought processes and meanings attributed to each question, the validity of the questionnaire items was tested to see how well these addressed the study objectives. This stage of the questionnaire testing also helped to identify any important omissions from the questionnaire items, and to record the length of time taken to complete the questionnaire.

The final questionnaire contained 29 items (summarised in Fig. 2) and was converted into electronic format using Qualtrics software (Provo, Utah). The survey, which took approximately ten minutes to complete, consisted of four sections. Sections 1 and 2 identified respondent demographics, experience levels and level of engagement in research generally. Section 3 explored in turn: respondents’ current surgical practice in relation to all acute proximal humeral fractures, respondents’ current treatment of patients with acute “low energy uncomplicated displaced fractures of the proximal humerus involving the surgical neck” and how much influence the PROFHER findings had on their practice. A final and optional section explored respondents’ general thoughts on what may influence changes to surgical practice. In order to improve usability and reduce the time taken to complete the questionnaire, skip logic was used throughout, so that certain items only appeared based on participants’ previous item responses. A progress bar, previous page facility, and ‘save and resume later’ function were also included to aid usability.

Fig. 2

Questionnaire items for this study (NIHR, National Institute for Health Research; PROFHER, Proximal Fracture of the Humerus: Evaluation by Randomisation; BESS, British Elbow and Shoulder Society).

Statistical analysis

Descriptive statistics were used to summarise the characteristics of the sample (Sections 1 and 2), in total and grouped by whether participants did or did not change practice based on the PROFHER findings. Quantitative responses regarding current and changed treatment practices (Section 3) were summarised descriptively in total and grouped by selected sample characteristics (including surgical and research experience).

Qualitative analysis

Free-text responses were analysed qualitatively using principles of Framework Analysis,⁹ whereby key concepts and recurrent themes were generated through an iterative process of familiarisation with the data, coding, and recoding until a thematic framework emerged. Three researchers (LJ, HH and SB) with expertise in different areas (qualitative research, orthopaedic research and surgical trial design) coded the data with regular discussion. Due to the large amount of qualitative data yielded, numerical counting of themes occurring in participants’ responses helped to explore trends.

Participant characteristics

Table I shows that most participants were aged between 36 and 55 years old (70.2%), were male (94.0%), worked in England (83.6%) and worked in a trauma unit (65.0%). Just over half were specialised shoulder surgeons (54.3%), mainly of consultant grade (82.9%) and with over ten years of experience in treating proximal humeral fractures (59.6%).

Survey results

Of the 265 participants included in the analysis, 137 (51.7%) had changed their practice in terms of surgical treatment of acute displaced proximal humeral fractures entirely (90 responses) or partly (47 responses) because of the PROFHER findings. In comparison, 128 participants (48.3%) did not change practice because of PROFHER, although three had changed practice for other reasons. However, of these 128 respondents, free-text responses revealed that 43 surgeons were already managing these types of patients non-operatively; for example: “I already practised along the lines of the PROFHER findings”.

Comparison of participants who did or did not change practice because of PROFHER

Surgeons who changed practice tended to be slightly younger, working in a trauma unit rather than a major trauma centre, be a shoulder surgeon rather than a trauma surgeon and have fewer years of experience in treating these fractures (Table I).

Table II summarises the clinicians’ prior involvement in research, direct involvement with the PROFHER trial and perceived validity and applicability of the PROFHER findings. Surgeons who changed their practice were more likely to have had a substantive role, mainly as Principal Investigator (PI), in an NIHR randomised surgical trial in the past (19.7% versus 8.7%). Overall, 39 participants (14.7%) had participated in the PROFHER trial, nine as PIs. Participation did not appear to be related to subsequent change in practice.

The majority of participants (245; 92.5%) were aware of the PROFHER findings before reading the research summary reported in the first page of the survey. Awareness was less in the group who did not change practice (86.7%). Curiously, three participants who reported to have changed practice because of PROFHER reported not to have been aware of it. Most participants heard about the trial results in meetings where these were presented (71.1%) and/or read them in one or more research publication (69.8%). Participants who changed their practice were more likely to have attended a meeting (78.4% versus 62.5%).

Participants who changed their surgical practice gave higher ratings (out of 10) to the validity of the PROFHER results (mean 7.4 versus 5.3) and the applicability of results to their own practice (mean 7.6 versus 5.6) than participants who did not change practice. There was greater variability in both scores among participants who did not change practice; this included all six instances of zero scores for both items.

Surgeons who did not change practice operated on a greater number of proximal humeral fractures (all types) and acute ‘uncomplicated’ displaced fractures of the proximal humerus involving the surgical neck (as eligible for the PROFHER trial) between April 2015 and March 2016 than those who did change practice (Table III). A low number of PROFHER-eligible fractures (frequency category 0 to 5) were operated on by 81.6% of surgeons who changed practice and 68.8% of surgeons who did not. None of the surgeons who changed practice operated on more than 15 PROFHER-eligible fractures.

The most popular surgical technique was fixation by locking plate (82.3% overall). Participants who changed practice were more likely to use reverse shoulder arthroplasty (45.3% versus 33.6%) and less likely to use intramedullary nails (27.0% versus 38.3%) than participants who did not change practice.

Participants who changed their practice were more likely to consider the PROFHER cost-effectiveness findings as “very influential” or “influential” than in those who did not (29.9% versus 7.3%).

A total of 38 respondents (15.0%) indicated there had been external changes that had impacted on their current practice. Statements supplied by 37 of these indicated two main categories. The first related to changes to the delivery of trauma services, usually reflecting local availability of specialist colleagues (18 responses). The second (12 responses) indicated service pressures (“very busy trauma lists”, bed and theatre availability issues) in various ways that impacted on surgical practice.

Qualitative results

The qualitative analysis of free-text data relating to change in practice was divided into groups of those participants who had or had not changed their practice because of PROFHER. The results of the fourth section of the survey, which explored wider factors affecting change in practice through free-text responses, are presented separately. In the following, we present the key emerging themes relating to these two aspects.

Respondents who had changed practice relating to PROFHER

All 137 respondents who indicated that they had changed practice provided further information about these changes through free-text responses. Most of these responses purely indicated a general move towards a more conservative approach. Where further information was offered, many highlighted how the PROFHER trial findings had given them greater confidence in making a decision to treat non-operatively (22 cases). Others suggested they had already been using conservative treatment for these patients (39 cases) and, in some instances, respondents commented that this had reduced concerns around liability as their practice was now evidence-based.

Several respondents indicated that they used the trial findings to inform a patient-centred approach, for example, through discussions of treatment options with patients in light of the trial findings (21 cases). Meanwhile, others commented on the need to review patients’ need for surgery on a case-by-case basis (24 cases), with specific exclusion criteria from the PROFHER trial being important in these decisions (e.g. intra-articular extension or associated dislocation). Although not an exclusion criteria in the PROFHER trial, age was a consideration for some respondents, who generally stated that they only considered operating on younger patients. In some instances, respondents gave examples of how their surgical approach had altered over recent years; for example, a reduction in the use of locking plates was observed.

The influence of wider opinion, such as colleagues’ change in practice, had apparently affected treatment decisions for some respondents. In some instances, respondents who were not specific shoulder specialists reported that the trial findings had led them to make fewer referrals to their specialist colleagues (nine cases). However, one respondent raised concerns that the trial findings may be misinterpreted by non-specialists, leading to reductions in referrals. Three respondents who had originally changed their practice in light of the PROFHER findings had reverted back to surgical management due to concerns around poorer outcomes from non-operative care; one indicating that “[the] study findings are not replicated in our population”.

Of those giving specific reasons as to how the PROFHER trial findings had prompted a change in practice, many were encouraged to change practice due to there being no difference in patient outcomes across treatment groups, as well as the greater costs and potential for harm associated with surgery. Several respondents commented on the definitive nature of the trial findings, which had provided high-quality evidence to support these decisions. In a handful of cases where respondents had “partly changed” their practice, respondents were negative about the trial results, citing concerns around validity, sample size or preferring to rely on ‘gut instinct’ when treating patients.

Respondents who had not changed practice relating to PROFHER

The responses of the 111 respondents providing further information as to why they had not changed their practice as a result of the PROFHER trial fell into three broad categories: those indicating that the respondents were already managing these patients conservatively (43 cases); those critical about the study (33 cases); and heterogeneous comments that could not be grouped into an overarching theme (35 cases). Of note was that, similar to the respondents who had changed their practice, respondents commented that the PROFHER results had given greater support when considering non-surgical management (19 cases) and had additionally aided discussions with patients (eight cases).

Of those 43 respondents indicating they were already managing patients conservatively, similar comments emerged as to the group who had indicated their practice had changed in recent years (described earlier). For example, many commented on the greater confidence in treating non-operatively now, some with fewer referrals to specialist colleagues. Likewise, several stressed the need to review patients on a case-by-case basis, with surgery being offered for fractures excluded from the PROFHER trial in four cases. Of the 33 expressing some criticism or reservation with the PROFHER trial, there were ten respondents who were highly critical of the study (4% of total sample). These responses corresponded to very low scores, typically 0, for the questionnaire items exploring perceived validity and applicability. Critical comments were mostly specific in nature, airing concerns about potential sources of bias such as selection bias (e.g. surgeons’ ability to exclude those patients with a clear indication for surgery), generalisability of the study findings (e.g. exclusion of complex fractures; lack of reverse shoulder arthroplasties), or concerns around study design (several relating to the pragmatic study design).

In terms of heterogeneous comments that were not grouped into one overarching theme, these comments related to respondents’ continuing reliance on their own judgement and professional experience that did not reflect the results of PROFHER and consequently led to a tendency for preferring surgical management.

General thoughts about factors influencing change in practice

Section 4 of the survey prompted free-text responses to three questions relating to factors influencing practice, with an option for further comments (Fig. 2). This optional section was completed by 136 respondents (51.1%); more of whom changed practice (56.2%, 77 of 137) than did not (47.7%, 59 of 129).

The need for an evidence base was the predominant response (110 cases) to the question asking “what factors are likely to influence your decision to change surgical practice?” The surgeons’ own experience (30 cases) and peers’ opinions (19 cases) were also expressed as important drivers for changing practice, with costs, choice of outcome measure used in research and needs of patients, including safety, featuring to lesser extents. For the second question, the majority considered that “practising EBM was becoming the norm in orthopaedic surgery”, either fully (52 cases), partly (15 cases) or at least on the way (23 cases), with several respondents pointing to limitations in the available evidence (17 cases). A total of 20 respondents considered that it was not the norm.

Several themes arose from the responses on what the respondent’s specialist society (e.g. BESS, BOA, British Society for Surgery of the Hand, Royal College of Surgeons) could do to guide change in their practice. One strong theme was the production of evidence-based guidelines, guidance (including standards) and summaries (including evidence updates and reviews) for information, to help and support decision-making (46 cases). This often linked with dissemination (24 cases), including at meetings in several cases. Several emphasised the need for critical appraisal. A second strong theme was society support, promotion, funding, and dissemination of research, mainly multicentre RCTs (29 cases), but also of other research that included epidemiology of conditions, national registries and identifying key clinical questions. Several directly endorsed the current activities, often specifically the support for large RCTs, of their specialist society (19 cases). There were a few exceptions, including those stressing the need to avoid the influence of industry.

Finally, when given the option of recording further comments, responses re-iterated the themes that had emerged mostly around the importance of the need for an evidence base and support for PROFHER, but also reservations about the limitations of the current evidence.