CLINICAL APPLICABILITY OF EVIDENCE-BASED ORTHOPEDICS: A CROSS-SECTIONAL STUDY OF THE QUALITY OF ORTHOPEDIC EVIDENCE

Abstract

Introduction: There is a general trend and even demand of using evidence-based methods in the practice of medicine. Especially in orthopedic surgery, which is a specialty traditionally employing treatments with obvious effectiveness, evidence-based clinical decision-making has become a strong trend. Yet all evidence-based decisions are only as sound as the evidence they are based on. In other studies, we could show that only 1 in 3 controlled orthopaedic trial accounts for confounding, and that there is even conflict in the results of meta-analyses, which are supposed to produce the highest level of evidence. This study asked how high the quality of evidence in orthopaedic research really is, and, thus, whether it would be applicable in “evidence”-based orthopaedics.

Method: All 2006 controlled trials from orthopedic journals with high impact-factor are analyzed in a cross-sectional study. A score based on the CONSORT statement was used to assess study quality. This score assesses power analyses, prospectiveness, randomization, allocation concealment and observer blinding, intention-to-treat, and how losses during follow-up were addressed. We also assessed whether there was an association between the test score and variables such as area of research and participation of a researcher with methodological training. Finally we tested the inter-observer reliability of our test between an investigator with postgraduate training in biostatistics and epidemiology and an orthopaedic resident with no specific methodological training beyond medical school.

Results: The overall quality of 126 studies was moderate to high, with an average score of 3.4 ± 1.7. The most neglected parameters were power analysis, intention-to-treat, and concealment. There were significant differences in results by area of research (p=0.022). The highest values were seen in oncology (4.6 ± 1.4) and osteoarthritis (4.4 ± 1.8), the lowest in traumatology (2.9 ± 1.3). The participation of a methodologically trained investigator increases study quality significantly (p=0.002). There is no difference in study quality whether there is statistically significant result or not (p=0.497). There was a 81.2% agreement, suggesting that study quality can be judged regardless of “specific training”.

Conclusion: We found good to high values for orthopaedic evidence on our scale, suggest high validity and applciability. We also saw that this instrument can be used without methodological training. However, there seems to be neglect of some important study features like power analyses, intention-to-treat, and concealment. Heightened awareness of this problem will help to increase the quality of orthopaedic evidence, and thus the clinical applicability of evidence-based orthopaedics.