Abstract
In recent literature, the fragility index (FI) has been used to evaluate the robustness of statistically significant findings of dichotomous outcomes. This metric is defined as the minimum number of outcome events to flip study conclusions from significant to nonsignificant. Orthopaedics literature is frequently found to be fragile with a median FI of 2 in 150 RCTs across spine, hand, sports medicine, trauma and orthopaedic oncology studies. While many papers discuss limitations of FI, we aimed to further characterize it by introducing the Fragility Likelihood (FL), a new metric that allows us to consider the probability of the event to occur and to calculate the likelihood of this fragility to be reached.
We systematically reviewed all randomized controlled trials in the Journal of Bone and Joint Surgery (Am) over 10 years. The FL was calculated with the following formula: A x B x C x 100% (A= FI; B = probability of the event in the group with the smallest number of events; C= probability of the non-event in the group with the highest number of events). A smaller FL demonstrates more robust results and conversely, a larger FL illustrates a higher likelihood of fragility being reached and more fragile the findings.
The median FI for the statistically significant outcomes was 2 (Mean: 3.8; Range 0-23). The median FL for the statistically significant outcomes was 11% (Mean: 22%, Range: 2%-73%). This means that the probability of reaching non-significance is only 11% when considering the probability of the event to occur. When comparing studies with the same FI we found the FL to range from 3% to 43%. This illustrates the large differences in robustness between trials with equal FI when the likelihood of the event was taken into consideration.
As orthopaedic studies are frequently reported as fragile, we found that by calculating the FL, studies may be more robust than previously assumed based off FI alone. By using the FL in conjunction with FI and p-values will provide additional insight into the robustness of the reported outcomes. Our results indicate that by calculating the FL, study conclusions are stronger than what the FI alone predicts. Although conducting RCTs in surgery can be challenging, we must endeavor to critically evaluate our results so we can answer important orthopaedic questions with certainty.
