Prospective multicenter external validation of postoperative mortality prediction tools in patients undergoing emergency laparotomy.
Our latest research was just published in the Journal of Trauma and Acute Care Surgery (American Association for the Surgery of Trauma), February 2, 2023.
Prospective multicenter external validation of postoperative mortality prediction tools in patients undergoing emergency laparotomy
by
Kokkinakis S, Kritsotakis EI, Paterakis K, Karali GA, Malikides V, Kyprianou A, Papalexandraki M, Anastasiadis, Charalampos S, Zoras O, Drakos N , Kehagias I, Kehagias D, Gouvas N ,Kokkinos G, Pozotou I, Papatheodorou P, Frantzeskou K, Schizas D, Syllaios A, Palios IM, Nastos K ,Perdikaris M, Michalopoulos NV, Margaris I, Lolis E, Dimopoulou G, Panagiotou D, Nikolaou V, Glantzounis GK,Pappas-Gogos G, Tepelenis K, Zacharioudakis G, Tsaramanidis S, Patsarikas I, Stylianidis G, Giannos G, Karanikas M, Kofina K, Markou M, Chrysos E, and Lasithiotakis K.
Access the full paper via:
Abstract
BACKGROUND
Accurate preoperative risk assessment in emergency laparotomy (EL) is valuable for informed decision-making and rational use of resources. Available risk prediction tools have not been validated adequately across diverse healthcare settings. Herein, we report a comparative external validation of 4 widely cited prognostic models.
METHODS
A multicenter cohort was prospectively composed of consecutive patients undergoing EL in 11 Greek hospitals from January 2020 to May 2021 using the National Emergency Laparotomy (NELA) audit inclusion criteria. 30-day mortality risk predictions were calculated using the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP), NELA, Physiological and Operative Severity Score for the enUmeration of Mortality and morbidity (P-POSSUM) and Predictive Optimal Trees in Emergency Surgery Risk (POTTER) tools. Surgeons' assessment of postoperative mortality using pre-defined cutoffs was recorded, and a surgeon-adjusted ACS-NSQIP prediction was calculated when the original model's prediction was relatively low. Predictive performances were compared using scaled Brier scores, discrimination and calibration measures and plots, and decision curve analysis. Heterogeneity across hospitals was assessed by random-effects meta-analysis.
RESULTS
631 patients were included and 30-day mortality was 16.3%. The ACS-NSQIP and its surgeon-adjusted version had the highest scaled Brier scores. All models presented high discriminative ability, with concordance statistics ranging from 0.79 for P-POSSUM to 0.85 for NELA. However, except the surgeon-adjusted ACS-NSQIP (Hosmer-Lemeshow test p = 0.742), all other models were poorly calibrated (p < 0.001). Decision curve analysis revealed superior clinical utility of the ACS-NSQIP. Following recalibrations, predictive accuracy improved for all models but ACS-NSQIP retained the lead. Between-hospital heterogeneity was minimum for the ACS-NSQIP model and maximum for P-POSSUM.
CONCLUSION
The ACS-NSQIP tool was most accurate for mortality predictions after EL in a broad external validation cohort, demonstrating utility for facilitating preoperative risk management in the Greek healthcare system. Subjective surgeon assessments of patient prognosis may optimise ACS-NSQIP predictions.
Figure 1. Calibration of prognostic models when predicting 30-day postoperative death. The smooth line is a smoothed locally weighted regression (lowess) line that shows the agreement between predicted probabilities and observed proportions of 30-day mortality. The dashed diagonal line indicates perfect calibration. The circled points represent mean risks in decile groups of predicted probabilities, with vertical lines representing 95% confidence intervals. The spike plot on the x-axis summarises the density of patients in the range of predicted risks of 30-day death. E:O, ratio of expected and observed deaths; CITL, calibration-in-the-large; Slope, calibration slope; AUC, area under the curve; GOF goodness-of-fit.
Figure 2. Decision curves showing the net benefit in clinical decision-making of using each prognostic model of 30-day postoperative mortality.
Figure 3. Summary forest plot with overall scaled Brier score for each prognostic model based on the results of random-effects meta-analysis of hospital-specific data. The confidence interval quantifies the precision in estimating the average Brier score in this study whereas the prediction interval quantifies the dispersion of the Brier score value in future single-center studies by accounting for between-hospitals heterogeneity.