Advertisement

The Statistical Fragility of Foot and Ankle Surgery Randomized Controlled Trials

  • Anthony J. Milto
    Affiliations
    Division of Orthopedics and Rehabilitation, Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL

    Center for Clinical Research, Southern Illinois University School of Medicine, Springfield, IL
    Search for articles by this author
  • Cecily E. Negri
    Affiliations
    Division of Orthopedics and Rehabilitation, Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL
    Search for articles by this author
  • Jeffrey Baker
    Affiliations
    Division of Orthopedics and Rehabilitation, Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL
    Search for articles by this author
  • Sowmyanarayanan Thuppal
    Correspondence
    Address correspondence to: Sowmyanarayanan Thuppal, MD, PhD, Southern Illinois University School of Medicine, 701 N First Street, PO Box 19679, Springfield, IL 62794-9679
    Affiliations
    Division of Orthopedics and Rehabilitation, Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL

    Center for Clinical Research, Southern Illinois University School of Medicine, Springfield, IL
    Search for articles by this author
Published:September 08, 2022DOI:https://doi.org/10.1053/j.jfas.2022.08.014

      Abstract

      Fragility index (FI) is a metric used to interpret the results of randomized controlled trials (RCTs), and describes the number of subjects that would need to be switched from event to non-event for a result to no longer be significant. Studies that analyze FI of RCTs in various orthopedic subspecialties have shown the RCTs to be largely underpowered and highly fragile. However, FI has not been assessed in foot and ankle RCTs. The MEDLINE and Embase online databases were searched from 1/1/2011 through 11/19/2021 for RCTs involving foot and ankle conditions. FI, fragility quotient (FQ), and difference between the FI and number of subjects lost to follow-up was calculated. Spearman correlation was performed to determine the relationship between sample size and FI. Overall, 1262 studies were identified of which 18 were included in the final analysis. The median sample size was 65 (interquartile range [IQR] 57-95.5), the median FI was 2 (IQR 1-2.5), and the median FQ was 0.026 (IQR 0.012-0.033). Ten of 15 (67%) studies with non-zero FI values had FI values less than the number of subjects lost to follow-up. There was linear association between FI and sample size (R2 = 0.495, p-value: .031). This study demonstrates that RCTs in the field of foot and ankle surgery are highly fragile, similar to other orthopedic subspecialties.

      Level of Clinical Evidence

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to The Journal of Foot and Ankle Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Wright JG
        • Swiontkowski MF
        • Heckman JD.
        Introducing levels of evidence to the journal.
        J Bone Joint Surg Am. 2003; 85: 1-3
        • Djulbegovic B
        • Guyatt GH.
        Progress in evidence-based medicine: a quarter century on.
        Lancet. 2017; 390: 415-423
        • Walsh M
        • Srinathan SK
        • McAuley DF
        • Mrkobrada M
        • Levine O
        • Ribic C
        • Molnar AO
        • Dattani ND
        • Burke A
        • Guyatt G
        • Thabane L
        • Walter SD
        • Pogue J
        • Devereaux PJ.
        The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index.
        J Clin Epidemiol. 2014; 67: 622-628
        • Constant M
        • Trofa DP
        • Saltzman BM
        • Ahmad CS
        • Li X
        • Parisien RL.
        The fragility of statistical significance in patellofemoral instability research: a systematic review.
        Am J Sports Med. 2021; ([Epub ahead of print])https://doi.org/10.1177/03635465211039202
        • Ehlers CB
        • Curley AJ
        • Fackler NP
        • Minhas A
        • Chang ES.
        The statistical fragility of hamstring versus patellar tendon autografts for anterior cruciate ligament reconstruction: a systematic review of comparative studies.
        Am J Sports Med. 2021; 49: 2827-2833
        • Ekhtiari S
        • Gazendam AM
        • Nucci NW
        • Kruse CC
        • Bhandari M.
        The fragility of statistically significant findings from randomized controlled trials in hip and knee arthroplasty.
        J Arthroplasty. 2021; 36 (.e1): 2211-2218
        • Evaniew N
        • Files C
        • Smith C
        • Bhandari M
        • Ghert M
        • Walsh M
        • Devereaux PJ
        • Guyatt G.
        The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey.
        Spine J. 2015; 15: 2188-2197
        • Forrester LA
        • Jang E
        • Lawson MM
        • Capi A
        • Tyler WK.
        Statistical fragility of surgical and procedural clinical trials in orthopaedic oncology.
        J Am Acad Orthop Surg Glob Res Rev. 2020; 4 ([Epub ahead of print])https://doi.org/10.5435/JAAOSGlobal-D-19-00152
        • Herndon CL
        • McCormick KL
        • Gazgalis A
        • Bixby EC
        • Levitsky MM
        • Neuwirth AL.
        Fragility index as a measure of randomized clinical trial quality in adult reconstruction: a systematic review.
        Arthroplast Today. 2021; 11: 239-251
        • Khan M
        • Evaniew N
        • Gichuru M
        • Habib A
        • Ayeni OR
        • Bedi A
        • Walsh M
        • Devereaux PJ
        • Bhandari M.
        The fragility of statistically significant findings from randomized trials in sports surgery: a systematic survey.
        Am J Sports Med. 2017; 45: 2164-2170
        • Khormaee S
        • Choe J
        • Ruzbarsky JJ
        • Agarwal KN
        • Blanco JS
        • Doyle SM
        • Dodwell ER.
        The fragility of statistically significant results in pediatric orthopaedic randomized controlled trials as quantified by the fragility index: a systematic review.
        J Pediatr Orthop. 2018; 38: e418-e423
        • Kyriakides PW
        • Schultz BJ
        • Egol K
        • Leucht P.
        The fragility and reverse fragility indices of proximal humerus fracture randomized controlled trials: a systematic review.
        Eur J Trauma Emerg Surg. 2021; ([Epub ahead of print])https://doi.org/10.1007/s00068-021-01684-2
        • Maldonado DR
        • Go CC
        • Huang BH
        • Domb BG.
        The fragility index of hip arthroscopy randomized controlled trials: a systematic survey.
        Arthroscopy. 2021; 37: 1983-1989
        • Marasco D
        • Russo J
        • Izzo A
        • Vallefuoco S
        • Coppola F
        • Patel S
        • Smeraglia F
        • Balato G
        • Mariconda M
        • Bernasconi A.
        Static versus dynamic fixation of distal tibiofibular syndesmosis: a systematic review of overlapping meta-analyses.
        Knee Surg Sports Traumatol Arthrosc. 2021; 29: 3534-3542
        • McCormick KL
        • Tedesco LJ
        • Swindell HW
        • Forrester LA
        • Jobin CM
        • Levine WN.
        Statistical fragility of randomized clinical trials in shoulder arthroplasty.
        J Shoulder Elbow Surg. 2021; 30: 1787-1793
        • Muthu S
        • Ramakrishnan E.
        Fragility analysis of statistically significant outcomes of randomized control trials in spine surgery: a systematic review.
        Spine (Phila Pa 1976). 2021; 46: 198-208
        • Parisien RL
        • Constant M
        • Saltzman BM
        • Popkin CA
        • Ahmad CS
        • Li X
        • Trofa DP.
        The fragility of statistical significance in cartilage restoration of the knee: a systematic review of randomized controlled trials.
        Cartilage. 2021; 13: 147S-155S
        • Parisien RL
        • Danford NC
        • Jarin IJ
        • Li X
        • Trofa DP
        • Vosseller JT.
        The fragility of statistical findings in Achilles tendon injury research: a systematic review.
        J Am Acad Orthop Surg Glob Res Rev. 2021; 5e21.00018https://doi.org/10.5435/JAAOSGlobal-D-21-00018
        • Parisien RL
        • Dashe J
        • Cronin PK
        • Bhandari M
        • Tornetta 3rd, P.
        Statistical significance in trauma research: too unstable to trust?.
        J Orthop Trauma. 2019; 33: e466-e470
        • Parisien RL
        • Trofa DP
        • Dashe J
        • Cronin PK
        • Curry EJ
        • Fu FH
        • Li X.
        Statistical fragility and the role of p values in the sports medicine literature.
        J Am Acad Orthop Surg. 2019; 27: e324-e329
        • Reito A
        • Raittio L
        • Helminen O.
        Fragility index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the fragility index in sports surgery.
        PeerJ. 2019; 7: e6813
        • Ruzbarsky JJ
        • Khormaee S
        • Daluiski A.
        The fragility index in hand surgery randomized controlled trials.
        J Hand Surg Am. 2019; 44: 698.e1-698.e7
        • Ruzbarsky JJ
        • Khormaee S
        • Rauck RC
        • Warren RF.
        Fragility of randomized clinical trials of treatment of clavicular fractures.
        J Shoulder Elbow Surg. 2019; 28: 415-422
        • Ruzbarsky JJ
        • Rauck RC
        • Manzi J
        • Khormaee S
        • Jivanelli B
        • Warren RF.
        The fragility of findings of randomized controlled trials in shoulder and elbow surgery.
        J Shoulder Elbow Surg. 2019; 28: 2409-2417
        • Checketts JX
        • Scott JT
        • Meyer C
        • Horn J
        • Jones J
        • Vassar M.
        The robustness of trials that guide evidence-based orthopaedic surgery.
        J Bone Joint Surg Am. 2018; 100: e85
        • Laflamme M
        • Belzile EL
        • Bédard L
        • van den Bekerom MP
        • Glazebrook M
        • Pelet S.
        A prospective randomized multicenter trial comparing clinical outcomes of patients treated surgically with a static or dynamic implant for acute ankle syndesmosis rupture.
        J Orthop Trauma. 2015; 29: 216-223
        • Li LH
        • Guo YZ
        • Wang H
        • Sang QH
        • Zhang JZ
        • Liu Z
        • Sun TS.
        Less wound complications of a sinus tarsi approach compared to an extended lateral approach for the treatment of displaced intraarticular calcaneal fracture: a randomized clinical trial in 64 patients.
        Medicine (Baltimore). 2016; 95: e4628
        • White TO
        • Bugler KE
        • Appleton P
        • Will E
        • McQueen MM
        • Court-Brown CM.
        A prospective randomised controlled trial of the fibular nail versus standard open reduction and internal fixation for fixation of ankle fractures in elderly patients.
        Bone Joint J. 2016; 98-b: 1248-1252
        • Svantesson E
        • Hamrin Senorski E
        • Danielsson A
        • Sundemo D
        • Westin O
        • Ayeni OR
        • Samuelsson K.
        Strength in numbers? The fragility index of studies from the Scandinavian knee ligament registries.
        Knee Surg Sports Traumatol Arthrosc. 2020; 28: 339-352
        • Potter GE.
        The fragility index should not be used for sample size calculations in clinical trials.
        J Clin Epidemiol. 2021; 142: 315-316
        • Condon TM
        • Sexton RW
        • Wells AJ
        • To MS.
        The weakness of fragility index exposed in an analysis of the traumatic brain injury management guidelines: A meta-epidemiological and simulation study.
        PLoS One. 2020; 15e0237879
        • Caldwell JE
        • Youssefzadeh K
        • Limpisvasti O.
        A method for calculating the fragility index of continuous outcomes.
        J Clin Epidemiol. 2021; 136: 20-25
        • Ahmad J
        • Ahmad SH
        • Jones K.
        Treatment of plantar fasciitis with botulinum toxin.
        Foot Ankle Int. 2017; 38: 1-7
        • An L
        • Zhang J
        • Ma W
        • Zhu L
        • He X
        • Ying J
        • Zhuang Y
        • Zhuang Y
        • Yu T.
        Clinical comparison of single-incision and dual-incision approaches for the treatment of distal tibial and fibular fractures: a randomized controlled trial.
        J Orthop Surg (Hong Kong). 2020; 282309499020930305https://doi.org/10.1177/2309499020930305
        • Bendtsen TF
        • Nielsen TD
        • Rohde CV
        • Kibak K
        • Linde F.
        Ultrasound guidance improves a continuous popliteal sciatic nerve block when compared with nerve stimulation.
        Reg Anesth Pain Med. 2011; 36: 181-184
        • Colcuc C
        • Blank M
        • Stein T
        • Raimann F
        • Weber-Spickschen S
        • Fischer S
        • Hoffmann R.
        Lower complication rate and faster return to sports in patients with acute syndesmotic rupture treated with a new knotless suture button device.
        Knee Surg Sports Traumatol Arthrosc. 2018; 26: 3156-3164
        • Georgiannos D
        • Lampridis V
        • Bisbinas I.
        Fragility fractures of the ankle in the elderly: open reduction and internal fixation versus tibio-talo-calcaneal nailing: short-term results of a prospective randomized-controlled study.
        Injury. 2017; 48: 519-524
        • Hauritz RW
        • Pedersen EM
        • Linde FS
        • Kibak K
        • Borglum J
        • Bjoern S
        • Bendtsen TF.
        Displacement of popliteal sciatic nerve catheters after major foot and ankle surgery: a randomized controlled double-blinded magnetic resonance imaging study.
        Br J Anaesth. 2016; 117: 220-227
        • Karaarslan S
        • Tekgul ZT
        • Simsek E
        • Turan M
        • Karaman Y
        • Kaya A
        • Gonullu M.
        Comparison between ultrasonography-guided popliteal sciatic nerve block and spinal anesthesia for hallux valgus repair.
        Foot Ankle Int. 2016; 37: 85-89
        • Kir MC
        • Ayanoglu S
        • Cabuk H
        • Dedeoglu SS
        • Imren Y
        • Karslioglu B
        • Yuce A
        • Gurbuz H.
        Mini-plate fixation via sinus tarsi approach is superior to cannulated screw in intra-articular calcaneal fractures: a prospective randomized study.
        J Orthop Surg (Hong Kong). 2018; 262309499018792742https://doi.org/10.1177/2309499018792742
        • Klammer G
        • Baumann G
        • Moor BK
        • Farshad M
        • Espinosa N.
        Early complications and recurrence rates after Kirschner wire transfixion in lesser toe surgery: a prospective randomized study.
        Foot Ankle Int. 2012; 33: 105-112
        • Laflamme M
        • Belzile EL
        • Bedard L
        • van den Bekerom MP
        • Glazebrook M
        • Pelet S.
        A prospective randomized multicenter trial comparing clinical outcomes of patients treated surgically with a static or dynamic implant for acute ankle syndesmosis rupture.
        J Orthop Trauma. 2015; 29: 216-223
        • Li LH
        • Guo YZ
        • Wang H
        • Sang QH
        • Zhang JZ
        • Liu Z
        • Sun TS.
        Less wound complications of a sinus tarsi approach compared to an extended lateral approach for the treatment of displaced intraarticular calcaneal fracture: a randomized clinical trial in 64 patients.
        Medicine (Baltimore). 2016; 95: e4628
        • Olsson N
        • Silbernagel KG
        • Eriksson BI
        • Sansone M
        • Brorsson A
        • Nilsson-Helander K
        • Karlsson J.
        Stable surgical repair with accelerated rehabilitation versus nonsurgical treatment for acute Achilles tendon ruptures: a randomized controlled study.
        Am J Sports Med. 2013; 41: 2867-2876
        • Sampath Kumar V
        • Marimuthu K
        • Subramani S
        • Sharma V
        • Bera J
        • Kotwal P.
        Prospective randomized trial comparing open reduction and internal fixation with minimally invasive reduction and percutaneous fixation in managing displaced intra-articular calcaneal fractures.
        Int Orthop. 2014; 38: 2505-2512
        • Sanders FRK
        • Birnie MF
        • Dingemans SA
        • van den Bekerom M
        • Parkkinen M
        • van Veen RN
        • Goslings JC
        • Schepers T.
        • RODEO Collaborator Group
        Functional outcome of routine versus on-demand removal of the syndesmotic screw: a multicentre randomized controlled trial.
        Bone Joint J. 2021; 103-B: 1709-1716
        • White TO
        • Bugler KE
        • Appleton P
        • Will E
        • McQueen MM
        • Court-Brown CM.
        A prospective randomised controlled trial of the fibular nail versus standard open reduction and internal fixation for fixation of ankle fractures in elderly patients.
        Bone Joint J. 2016; 98-B: 1248-1252
        • Winge R
        • Ryge C
        • Bayer L
        • Klausen TW
        • Gottlieb H.
        Wound complications after ankle surgery. Does compression treatment work? A randomized, controlled trial.
        Eur J Trauma Emerg Surg. 2018; 44: 947-956
        • Yu T
        • Ying J
        • Liu J
        • Huang D
        • Yan H
        • Xiao B
        • Zhuang Y.
        Percutaneous posteroanterior screw fixation for Haraguchi type 1 posterior malleolar fracture in tri-malleolar fracture: operative technique and randomized clinical results.
        J Orthop Surg (Hong Kong). 2021; 292309499021997996https://doi.org/10.1177/2309499021997996
        • Zhang J
        • Ebraheim N
        • Lause GE
        • Xiao B
        • Xu R.
        A comparison of absorbable screws and metallic plates in treating calcaneal fractures: a prospective randomized trial.
        J Trauma Acute Care Surg. 2012; 72: E106-E110