Purpose: Unbiased outcome assessment in orthopedic clinical trials has the potential to improve trial validity. The approaches used to limit bias in outcome assessment in orthopaedic trials remain unclear. The objective of this systematic review was to assess the reporting and process of outcomes assessment practices in the current orthopaedic trauma literature.

Method: We searched eight high-impact-factor medical and orthopaedic journals manually and using the MED-LINE electronic database for reports of randomized controlled trials published from 2005 to 2008 pertaining to the surgical treatment of trauma-related injuries. Two reviewers independently determined study eligibility and extracted relevant data from included trials.

Results: Of the 7910 citations identified during our search, 47 randomized controlled trials, which included a total of 4706 patients, met our inclusion criteria. Of 47 studies, 39 (83%) provided a statement to describe some process of outcome assessment and 29 (74%) reported using an unblinded individual as the outcome adjudicator. Four studies (10%) reported using a second assessor to verify outcome measurements, and three studies (8%) reported the use of an adjudication committee to reach endpoint decisions via consensus. No included study provided a rationale for the use of their chosen approach to adjudication. The most commonly adjudicated outcomes included fracture healing (15 studies), reoperation rate (6 studies), and general clinical assessment of post-operative complications and limb function (30 studies), mainly by orthopaedic surgeons. Blinding of outcome assessors was not performed or unclear in 38 studies (81%).

Conclusion: Despite the importance of the outcome assessment process in orthopedic trauma trials, key aspects of outcome assessment are insufficiently reported. This limits the ability of readers to assess the validity of published trials.

Purpose: Although blinding of outcome assessors is crucial to minimize bias in clinical trials, the majority of surgical trials do not blind these individuals in part due to practical difficulties inherent in surgical interventions. We devised and tested techniques to blind outcome assessors in trials of femoral neck fracture fixation.

Method: We developed three techniques to mask radiographs of femoral neck fractures fixated with cancellous screws or dynamic hip screws: Blackout, Subtraction, and Overlay. 50 orthopaedic trauma surgeons assessed 32 radiographs blinded with each of these techniques. We considered:

The ability to mask the surgeons (the proportion of radiographs in which the surgeons were able to correctly identify the implant and the Bang Blinding Index);

Results: All three techniques achieved low proportions of correct identification of cancellous or dynamic screws (14.9% for Blackout, 26.9% for Subtraction, 22.1% for Overlay) and high proportions of “don’t know” responses (72.3%, 48.4%, 52.8% respectively). The Bang Blinding Indices were close to 0 (perfect blinding) for all three techniques (−0.024 to 0.008). The interrater reliability of quality of reduction in the blinded images (ICC = 0.55 – 0.57) was similar to the reliability of the unblinded radiographs (ICC = 0.60). Surgeons perceived the Overlay images as much more difficult to rate in 6.9% of radiographs, compared with 9.7% of Subtraction images (p=0.25) and 28.0% of Blackout images (p< 0.001).

Conclusion: Three techniques of blinding radiographs of femoral neck fractures successfully mask surgeons to the type of implant fixated, do not compromise reliability of reduction ratings, and do not make the rating process substantially more difficult. Trialists should explore creative approaches such as these to blind as many individuals as possible when designing trials, and should incorporate rigorous approaches to testing the success of blinding.