Abstract
Objectives: An evidence-based trauma database that rated the level-of-evidence for the majority of orthopedic trauma literature would be useful to orthopedic trauma surgeons and researchers. Currently, the development of a database using evidence ratings for such a large body of literature is a time-consuming and expensive endeavor. However, if an accurate rating could be obtained from an abstract without reading the full text, such an endeavor would be feasible. Thus, the purpose of this study is to determine if a reviewer can successfully ascertain the level-of-evidence from the article’s abstract without reading the full text article.
Methods: We compared the level-of-evidence rating obtained from reading an article’s full text with the rating obtained from reading the abstract alone. We reviewed 162 clinical articles published from January 2000 through April 2004 in five orthopedic trauma journals. To establish a full text rating, two reviewers independently rated the full text of each article for study type (therapeutic, prognostic or diagnostic) and level-of-evidence (I through V). The ratings from the two reviewers were compared and disagreements were reconciled through discussion to form a final full text rating. A third reviewer rated the abstracts of each article without access to the full text or full text rating.
Results: Of the 162 articles, 118 (73%) were therapeutic, 40 (25%) were prognostic and 4 (3%) were diagnostic. Most studies represented level IV evidence (65%), with only 15% representing the highest level-of-evidence, level I. Kappa values for agreement between the two reviewers rating the full text were 0.81 for study type and 0.79 for level-of-evidence, and the kappa values for agreement between the final full text rating and the abstract rating were 0.68 for study type and 0.75 for level-of-evidence, respectively. Thirteen of the discordant abstracts (8% of the entire set) lacked sufficient or correct evidence compared with the full text to render an accurate rating.
Conclusion: Accurate study type and level-of-evidence rating in trauma articles can be obtained in most orthopedic trauma articles from reading an article’s abstract alone. Thus, developing an evidence-based trauma database that uses abstracts to rate the level-of-evidence of the orthopedic trauma literature appears feasible and appropriate.
This research was supported by a grant from the AO Foundation, Clinical Investigation and Documentation, Clavadelerstrasse, 7270 Davos Platz, Switzerland
Correspondence should be addressed to Ms Larissa Welti, Scientific Secretary, EFORT Central Office, Technoparkstrasse 1, CH-8005 Zürich, Switzerland
