PAPER 157: WHAT AUTHORS DON’T TELL US? THE QUALITY OF REPORTING OF ORTHOPAEDIC RANDOMIZED TRIALS USING A CHECKLIST FOR NON-PHARMACOLOGICAL THERAPIES

Abstract

Purpose: The CONSORT statement for the reporting of randomized controlled trials (RCTs) has limitations in its applicability to non-pharmacological trials. In response, the CLEAR NPT, a checklist that provides guidelines for the reporting of RCTs in surgery, has been developed. We aimed to

1. apply the CLEAR NPT to orthopaedic RCTs across multiple journals from 2004–2005, and

2. survey authors when items in the CLEAR NPT were not reported to determine if they were actually conducted.

We hypothesized that “lack of reporting” did not necessarily correlate with “not being conducted”.

Method: We searched for relevant orthopaedic RCTs across eight journals (four Orthopaedic, four General Medicine) in the period from January 2004 through December 2005. We applied the CLEAR NPT to all eligible studies. We contacted authors to determine what methodological safeguards were actually used, especially when details remained unclear from the publication.

Results: We included eighty-seven RCTs from eightyfive scientific reports. In assessing the RCTs with the CLEAR NPT, seventy-three (84%) studies had unclear reporting of allocation concealment. Only seventeen (20%) studies made mention of operator skill or experience. Participant, ward staff, rehabilitation staff, clinical outcome assessor and non-clinical outcome assessor blinding were found to be unclear in forty-eight (55%), sixty-three (72%), sixty-four (74%), forty (46%) and thirty-three (38%) studies respectively. Authors from forty-three RCTs responded to our survey. In direct contact, authors reported adequate allocation concealment 41% (95% CI = 25–58%) of the time when this was unclear from the RCT report. 70% of authors acknowledged that they had set objective measures such as minimum case criteria and/or comparison to good clinical outcomes. Authors specified that they had blinded relevant groups 28–40% of the time, despite unclear reporting in the publications.

Conclusion: The quality of reporting in the orthopaedic literature was highly variable. Readers should not assume that bias reducing safeguards not reported in an RCT did not occur. Our study reinforces the need for a tool like the CLEAR NPT to assess the methodology of surgical trials.