A SYSTEMATIC REVIEW OF CONFLICTING META-ANALYSES IN ORTHOPEDIC SURGERY

Abstract

Introduction: Meta-analyses are an important instrument in orthopaedic surgery, not only to create clinical guidelines, but also because their findings are included in public health and health policy decision-making. Generally, meta-analyses of randomised controlled trials are considered as the highest level of evidence. However, with increasing numbers of meta-analyses, discordance and frank conflicts in results have been seen, which might lead to grave complications considering the aforementioned facts. The purpose of this study was to search for conflicting meta-analyses in orthopaedic surgery, i.e. such arriving at different conclusions despite following the same research question; to identify potential reasons for, and to assess the actual amount and significance of such differences.

Methods: We searched the online databases PubMed, EMBASE and the Cochrane Controlled Trial Register for orthopaedic meta-analyses and cross-referenced results within and across databases to identify meta-analyses focusing on the same subject. Meta-analyses were defined as conflicting if they arrived at different results despite studying the same populations.

To assess the significance of such difference we used Cochrane’s Q-test. To test the amount, thus clinical meaning, of differences we calculated the I2-index, the amount of difference beyond random chance. Since both these parameters depend on study size, we also calculated the “uncertainty interval” (UI), which, in accordance to the 95% confidence interval contains the true I2-index of the whole population.

Results: We were able to identify conflicting meta-analyses on graft choice in ACL reconstruction (n=7), the use of hyaluronic acid (n=5) and pulsed electromagnetic fields in osteoarthritis (n=2). Significant differences could only be shown among meta-analyses on hyaluronic acid (p< 0.001). The uncertainty intervals were 38.6% to 78.6% for hyaluronic acid, 0% to 41.1% for ACL and 0% to 99% for electromagnetic fields in osteoarthritis.

Discussion: There are conflicting meta-analyses in orthopaedic research, posing a threat to evidence-based treatments. It seems, however, that a considerable amount of conflict derives from differences in the interpretation of pooled results rather than from the results themselves. In summary, findings and interpretations of meta-analyses should be as critically scrutinized as in any other type of study and subjected to re-assessment if deemed necessary.