Abstract
Background
As the number of studies in the literature is increasing, orthopaedic surgeons rely heavily on meta-analyses as their primary source of scientific evidence. The objectives of this review were to assess the scientific quality and number of published meta-analyses on orthopaedic-related topics over time.
Methods
We conducted, in duplicate and independently, a systematic review of published meta-analyses in orthopaedics in the years 2005 and 2008 and compared them with a previous systematic review of meta-analyses from 1969-1999. A search of electronic databases (Medline, EMBASE, and the Cochrane Database of Systematic Reviews (CDSR)) was performed to identify meta-analyses published in 2005 and 2008. We searched bibliographies and contacted content experts to identify additional relevant studies. Two investigators used the Oxman and Guyatt Index to assess the quality of the studies and abstracted relevant data.
Results
We included 45 and 44 meta-analyses from 2005 and 2008, respectively. While the number of meta-analyses increased five-fold from 1999 to 2008, the mean quality score did not change significantly over time (p=0.067). A significantly lower proportion of meta-analyses had methodological flaws (56% in 2005 and 68% in 2008) compared to meta-analyses published prior to 2000 (88%) (p=0.006). In 2005 and 2008, respectively 18% and 30% of meta-analyses had major to extensive flaws in their methodology. Studies from 2008 with positive conclusions did not use and report appropriate criteria for the validity assessment as often as those reporting negative results. The use of random-effects and fixed-effects models as pooling methods became more popular toward 2008.
Conclusion
Although methodological quality of orthopaedic meta-analyses has increased in the past 20 years, a substantial proportion displays major to extensive methodological flaws. As the number of published meta-analyses is increasing, a routine checklist for scientific quality should be used in the peer-review process to ensure methodological standards for publication.
