Interpretable prediction of 30-day mortality in patients with acute pancreatitis based on machine learning and SHAP

Abstract

Background

Severe acute pancreatitis (SAP) can be fatal if left unrecognized and untreated. The purpose was to develop a machine learning (ML) model for predicting the 30-day all-cause mortality risk in SAP patients and to explain the most important predictors.

Methods

This research utilized six ML methods, including logistic regression (LR), k-nearest neighbors(KNN), support vector machines (SVM), naive Bayes (NB), random forests(RF), and extreme gradient boosting(XGBoost), to construct six predictive models for SAP. An extensive evaluation was conducted to determine the most effective model and then the Shapley Additive exPlanations (SHAP) method was applied to visualize key variables. Utilizing the optimized model, stratified predictions were made for patients with SAP. Further, the study employed multivariable Cox regression analysis and Kaplan-Meier survival curves, along with subgroup analysis, to explore the relationship between the machine learning-based score and 30-day mortality.

Results

Through LASSO regression and recursive feature elimination (RFE), 25 optimal feature variables are selected. The XGBoost model performed best, with an area under the curve (AUC) of 0.881, a sensitivity of 0.5714, a specificity of 0.9651 and an F1 score of 0.64. The first six most important feature variables were the use of vasopressor, high Charlson comorbidity index, low blood oxygen saturation, history of malignant tumor, hyperglycemia and high APSIII score. Based on the optimal threshold of 0.62, patients were divided into high and low-risk groups, and the 30-day survival rate in the high-risk group decreased significantly. COX regression analysis further confirmed the positive correlation between high-risk scores and 30-day mortality. In the subgroup analysis, the model showed good risk stratification ability in patients with different gender, renal replacement therapy and with or without a history of malignant tumor, but it was not effective in predicting peripheral vascular disease.

Conclusions

the XGBoost model effectively predicts the severity of SAP, serving as a valuable tool for clinicians to identify SAP early.

Introduction

Acute pancreatitis (AP) is an inflammatory disease of the pancreas whose incidence and hospitalization rates have increased in recent years, affecting more than 3 million U.S. patients annually [1]. Variables that influence disease severity in AP people include comorbidities and demographic variables such as older age, type 2 diabetes [2], cardiovascular disease, kidney disease [3] and obesity [2, 4]. According to statistics, nearly 25% of AP patients develop SAP due to serious complications and have to be transferred to the intensive care unit (ICU) for treatment and the mortality rate of SAP patients is as high as 30% [5,6,7]. This imposes a huge health and economic burden on patients and society. Therefore, early risk assessment and timely treatment of patients with AP are very important to improve their clinical outcomes.

Many researchers have tried to evaluate and predict the severity and clinical prognosis of AP with some laboratory tests, scoring models, predictive models, etc., but they have different limitations. For example, C-reactive protein (CRP) levels were significantly higher in SAP patients 48 h later, but less accurate at admission [8]. The Acute Physiological and Chronic Health Assessment (APACHE II) score was primarily designed to assess critically ill patients rather than AP patients [9]. The Acute Pancreatitis Severity Bedside Index (BISAP) was used primarily to identify the severity of AP and its mortality, while the Harmless Acute Pancreatitis Score (HAPS) was more sensitive and accurate in identifying mild AP [10, 11]. In addition, the CT score is often used to assess the severity of AP, but it mainly focuses on the qualitative assessment of local injuries in and around the pancreas, ignoring important clinical symptoms, signs and biochemical indicators [12]. A study of the accuracy of existing clinical scoring models by Mounzer et al., suggests that existing methods may not meet the need to accurately predict SAP risk [13]. Therefore, it is necessary to develop a new model with high accuracy to predict SAP risk.

Currently, machine learning (ML) algorithms are increasingly being used to solve medical problems, building models based on training data sets that can improve risk prediction [14, 15] and drug interactions [16, 17] for a variety of diseases, including AP. Compared with traditional logistic regression and linear regression, ML models produce more stable predictions due to their advantages of higher-order nonlinear interactions. Most previous studies have used ML models to predict the severity of AP and fewer focus on predicting clinical outcomes of ICU patients. SHAP can explain the contribution of variables in an ML model to predict risk, making up for the unexplainable shortcomings of ML [18]. This study aims to develop a prognostic model for SAP patients by combining multiple ML algorithms and SHAP values to provide a powerful tool for clinical decision-making. To the best of our knowledge, this is the first study to apply explicable ML to predict clinical outcomes for SAP patients in the ICU.

Methods

Study population

This study utilized the Medical Information Mart for Intensive Care (MIMIC)-IV database for analysis. The database is a public resource that compiles information on patients hospitalized in the ICU of Beth Israel Deaconess Medical Center during 2001–2012. Access to the database was granted by the Massachusetts Institute of Technology (MIT) and Beth Israel Deaconess Medical Center, and the collection of the original data was obtained with consent. Patient information included in the MIMIC-IV database is anonymous; accordingly, informed consent was not required.

This study included AP patients who met the International Classification of Diseases, Ninth Revision (ICD-9) code of 577.0, were over 18 years of age, and had an ICU stay of more than 24 h. If a single patient had multiple ICU admission records, only data related to the first ICU admission were analyzed.

The study data was extracted from the raw data using Structured Query Language (SQL) with DataGlip (v 2021.2.1) and further processed in R (v 4.4.0, the R Foundation for Statistical Computing) for retrieval of subject information from the database. Baseline characteristics within 24 h of hospital entry were captured.

Data collection

The data analyzed in this study encompassed demographic characteristics, including age, sex, and race, along with vital signs recorded within the first 24 h of admission. These vital signs comprised temperature, heart rate, respiratory rate, blood pressure, and oxygen saturation (SpO2), with the average, minimum, and maximum values of each parameter being documented from multiple measurements within this time frame. Laboratory results were also examined, specifically white blood cell (WBC) count, hemoglobin, platelet count, serum creatinine, albumin, bilirubin, calcium, potassium, and lactic acid levels, recording their average, minimum, and maximum values over 24 h. Additionally, various clinical scores were noted, including SOFA (Sequential Organ Failure Assessment), SIRS (Systemic Inflammatory Response Syndrome), SAPS III (Simplified Acute Physiology Score III), OASIS (Oxford Acute Severity of Illness Score), and the Charlson co-morbidity index. Further data collection encompassed the presence of septicemia, myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular disease, dementia, chronic lung disease, rheumatic disease, peptic ulcer disease, diabetes, liver disease, paraplegia, malignant tumors, metastatic solid tumors, acute respiratory distress syndrome (ARDS), and acute kidney injury (AKI) stage. The primary endpoint was all-cause mortality within 30 days.

In this modeling analysis, categorical variables such as gender, past medical history, and treatment modalities were transformed into binary numeric formats. Gender was coded as 1 for males and 0 for females. For past medical history, conditions including Myocardial infarction, Congestive Heart Failure, and Peripheral Vascular Disease were coded as 1 if present, and 0 otherwise. Treatment variables involving the use of pressor drugs, diuretics, sedatives, and Continuous Renal Replacement Therapy (CRRT) were similarly coded as 1 when utilized and 0 when not. This binary coding method facilitates the effective integration of categorical variables into statistical analyses.

Data processing

Missing values are frequently encountered in MIMIC-IV databases. When the proportion of missing data for a variable exceeds 30%, that variable is excluded from further analysis. Conversely, if the proportion of missing data is less than 30%, the “mice” package (version 4.1.2) [19] implemented in R is employed to perform multiple imputations, thereby minimizing bias. Specifically, variables with a normal distribution are imputed using mean interpolation, while those with non-normal distributions are addressed through median interpolation.

Feature selection

First, we used RFE based on five-fold cross-validation to select features from the train set [20]. In the study, the “mlbench” (v 2.1-5) and “caret” (v 6.0–94) packages within the R programming environment were utilized to perform feature selection on training datasets via RFE with five-fold cross-validation. This iterative RFE method builds a model to identify and remove the most significant features, then reassesses the remaining features until all have been evaluated, aiming to identify an optimal feature subset. The five-fold cross-validation ensures that each subset of the original dataset is used once as validation data, facilitating robust model training. Performance metrics such as accuracy are calculated in each iteration, following the removal of the least important features, to evaluate the efficacy of the reduced feature set. Through this methodical process, the most effective subset of characteristics is ascertained, enhancing the predictive accuracy of the modeling approach.

Besides, in this study, we also employed the LASSO method for variable selection with the ″glmnet″ package (v 4.1-8). LASSO regression is a shrinkage estimation method used to address multicollinearity between covariates. When multiple correlated predictors are present, LASSO selects one and ignores others or sets some regression coefficients to zero. It is worth pointing out that the λ value is determined when the cross-validation error is within one standard error (SE) of its minimum because LASSO regression uses cross-validation to select the λ value based on the 1-SE criterion. We obtained a subset of features selected by LASSO.

Finally, we take the intersection of the two selection results to obtain the final subset of features.

Model establishment and evaluation

499 patients were grouped into a train and test set in a 4:1 ratio by stratified random sampling. The training set was preprocessed using a synthetic minority oversampling technique combined with an edited nearest neighbor (SMOTE + ENN) technique to balance the positive and negative classes [21]. This preprocessing was executed using the ″smotefamily″ package (v 1.4.0), which includes SMOTE in R. And then, based on the training dataset, we established six ML models, including LR, KNN using the ″kknn″ package(v 1.3.1), SVM with the ″e1071″ package (v 1.7–14), NB also through ″e1071″ package (v 1.7–14), RF using the″ randomForest″ package (v 4.7–1.1) and XGBoost utilizing the ″xgboost″ package (v 1.7.7.1) for predicting 30-day all-cause mortality in SAP patients [22]. The hyperparameters of these ML models were optimized using the quintuple cross-validation method provided by the Grid Search algorithm implemented via the ″caret″ package (v 6.0–94) in R.

Subsequently, we evaluated and compared the performance of each model in the test set. To avoid bias and overfitting and obtain more stable predictive performance, we repeated these ML methods 100 times with different random seeds and computed the average performance over these 100 repeats [23]. Finally, multiple indicators including AUC, sensitivity, specificity and F1 score were comprehensively evaluated, and the ″pROC″ package (version 1.18.5) was utilized to compute the AUC.

Model explanation

In this study, we employed SHAP as a method of interpretability to enhance the transparency of our predictive model. SHAP is recognized for its post-hoc analytical capacity to quantify the impact of individual features on the output of the model both individually and collectively, thereby clarifying the model’s operational mechanisms [18, 24]. Specifically, it calculates the Shapley value for each attribute of a data point using specific algorithms, indicating that feature contributions are additive. This approach facilitates comprehensive explanations of how each feature influences the predictive accuracy and likelihood in each data set.

Unlike traditional Feature Importance metrics commonly associated with many machine learning models, SHAP analysis offers greater statistical depth and interpretability, as evidenced by several prior studies [25, 26]. Therefore, to elucidate the decision-making processes underlying our model, we implemented the SHAP methodology. In this investigation, the ″xgboost″ package (v 1.7.7.1) alongside SHAP analysis was utilized to ascertain the critical predictors of 30-day all-cause mortality in patients with acute pancreatitis. This approach effectively highlighted the most impactful variables within the model.

The calculation formula of SHAP is as follows:

$$\:{\varphi\:}_{i}\left(f\right)=\sum\:_{S\subseteq\:N\setminus\:\left\{i\right\}}\:\frac{\left|S\right|!\left(\right|N|-|S|-1)!}{\left|N\right|!}\left(f(S\cup\:\{i\left\}\right)-f\left(S\right)\right)$$

where:

• $\:{\varphi\:}_{i}\left(f\right)$ is the SHAP value for feature $\:i$, quantifying its contribution to the prediction.

• $\:N$ is the set of all features included in the model.

• $\:S$ is any subset of the features that does not include feature $\:i$.

• $\:f\left(S\right)$ is the model output utilizing the features in subset $\:S$.

• $\:f(S\cup\:\{i\left\}\right)$ is the model output when feature $\:i$ is added to the subset $\:S$.

• $\:\left|S\right|!$ is the factorial of the number of elements in subset $\:S$, accounting for permutations of features within $\:S$.

• $\:\left(\right|N|-|S|-1)!$ is the factorial representing the number of features not in subset $\:S$ excluding the feature $\:i$.

• The factor $\:\frac{\left|S\right|!\left(\right|N|-|S|-1)!}{\left|N\right|!}$ normalizes the influence of subsets of various sizes by accounting for the number of possible permutations of the features in and out of $\:S$, ensuring a balanced contribution from all subsets.

Model predictions

In the predictive segment of the study, patients in train set were stratified into low-risk and high-risk groups based on the optimized Jordan index to assess the predictive capability of the model. The evaluation was conducted using the following strategies:

(1) Log-rank Test: The variance of Kaplan-Meier survival curves was analyzed using the log-rank test to determine if the differences in survival time distributions between the different risk groups were statistically significant. This statistical testing was conducted using the ″survival″ package (v 3.6-4) in R.

(2) Multivariate Cox Regression Analysis: This analysis assessed the correlation between the risk classification predicted by the machine learning model and 30-day all-cause mortality. It also considered potential confounding factors to ascertain the independent impact of risk prediction on forecasting the short-term risk of mortality. The analysis was performed using the ″survival″ package (v 3.6-4) and the ″forestplot″ package (version 3.1.3).

(3) Subgroup Analysis: The predictive effectiveness of the machine learning model was further evaluated in various subgroups, including sex, age, artificial renal replacement therapy, peripheral vascular disease, cerebrovascular disease, and malignant tumor. This analysis aimed to identify specific patient characteristics or clinical conditions that may influence the accuracy and reliability of the model’s predictions. The ″forestplot″ package (version 3.1.3) was utilized to visualize the results and interactions within the subgroups.

This analysis aimed to identify specific patient characteristics or clinical conditions that may influence the accuracy and reliability of the model’s predictions.

Statistical analysis

Continuous data were depicted as mean ± standard deviation (SD) or median (interquartile range [IQR]) and compared with Student’s t test or Mann-Whitney U test. Categorical variables were expressed as numbers (percentages) and compared using the chi-square test. The normality of data distribution was evaluated with the Shapiro-Wilk test. Non-normally distributed data or data exhibiting heterogeneity of variance were compared using the Kruskal-Wallis or Mann-Whitney U test. The P-value < 0.05 was deemed as statistically meaningful and the statistical analysis was conducted with R (v 4.4.0).

Results

Patient characteristics

The flowchart was shown in Fig. 1. Initially, the dataset included data from 1,069 patients.we applied specific inclusion criteria: patients were admitted to the ICU for the first time, aged between 17 and 90 years, and stayed in the ICU for more than 24 h. This refinement process narrowed the number of suitable patients down to 499, including 300 men (60.1%) and 199 women (39.9%). At 30 days follow-up, 74 patients (14.8%) died. 499 patients were separated into a train set (399) and a test set (100). The general characteristics of the patients were presented in Table 1. There is no significant difference between the training dataset and the test dataset except for the variables “Potassium” and “Aniongap”. Screened characteristics differed between fatal and nonfatal subjects in the training cohort (Table 2).

Fig. 1

The flowchart of this study

Table 1 The basic demographic characteristics of all patients

Table 2 Differences in screening characteristics between dead and non-dead subjects in the training group

Feature selection

The initial dataset comprised 230 variables. Due to the presence of missing values, 87 of these variables were discarded, leaving 143 variables available for further analysis. By employing the LASSO method with an optimal lambda value of approximately 0.0127, the number of significant variables was reduced to 47. These attributes demonstrated minimal errors in predictive modeling. Additionally, recursive feature elimination (RFE) is used to identify another set of 53 important features based on accuracy:0.8547, Kappa:0.04696. A subsequent comparison and intersection of the attributes selected by both LASSO and RFE methods further refined this to the 25 most pertinent attributes, which were used to enhance the performance of the machine learning models (Fig. 2).

Fig. 2

Feature selection. A: Changes in the Coefficients of 104 Predictive Variables with Variations in the Regularization Parameter λ; B: Feature selection in Lasso regression analysis: The horizontal axis represents the change of λ, the vertical axis shows the coefficient of each characteristic variable; C: Features selected by RFE with the 5-fold Cross-Validation: This graph demonstrates the prediction accuracy of different feature sets selected through Recursive Feature Elimination (RFE) using 5-fold cross-validation. The horizontal axis displays the number of features included in the model, while the vertical axis represents the accuracy achieved by each subset; D: Venn Diagram Showing the Intersection Between Lasso Regression and Recursive Feature Elimination (RFE) with Five-Fold Cross-Validation

The 25 features included age, temperature_mean, mbp_max, sbp_max, SPO₂_max, SPO₂_min, SPO₂_mean, Charlson Comorbidity Index, APSIII, bun_min, anion gap_min, wbc_min, mbp_min, bilirubin_total, alp_min, glucose_min, rdw_max, PTT_min, rheumatic disease, metastatic solid tumor, peripheral vascular disease, myocardial infarct, malignant cancer, continuous renal replacement therapy (CRRT) and vasopressor.

Model hyper-parameters

Utilizing a dataset composed of 25 variables, advanced modeling techniques such as LR, SVM, RF, Random Forest (RF), and XGBoost were implemented. The training set was preconditioned using the SMOTE + ENN to ensure a balanced representation of positive and negative classifications. A grid search method was employed to identify the most effective hyperparameters for each model. Details regarding the adjustment of these parameters are thoroughly outlined in Supplementary Table S1. Parameters not specifically adjusted adhered to their default settings.

Performance of ML models

The analysis involved training and evaluating six prevalent machine learning models: LR, KNN, NB, RF, SVM and XGBoost, as detailed in Fig. 3; Table 3.

Fig. 3

AUC Comparison for Different Models: A: Logistic Regression (LR); B: K-Nearest Neighbors (KNN); C:Naive Bayes (NB); D:Random Forests (RF); E:Support Vector Machines (SVM); F: Extreme Gradient Boosting (XGBoost)

Table 3 Results of ML modeling of 30-day follow-up mortality in SAP patients

The evaluation metrics included sensitivity, specificity, recall, accuracy, F1 score and AUC. As per the results in Fig. 3; Table 3, the XGBoost model outperformed the other models with an AUROC of 0.881, an accuracy of 0.91, and an F1 score of 0.64. In terms of sensitivity, which is identical to the recall rate, Naive Bayesian and SVM demonstrated commendable performance, both registering a value of 0.6429, whereas KNN lagged with a rate of 0.4286. RF showcased the highest specificity at 0.9651, outperforming other models, which ranged between 0.8721 and 0.9302 in this metric. Concurrently, XGBoost attained the highest accuracy, whereas KNN scored the lowest at 0.81.

Overall, the data illustrates that the XGBoost model offers superior predictive capabilities. Conversely, KNN’s overall effectiveness was comparatively lower as indicated in Fig. 3; Table 3.

In conclusion, given its robust performance across various metrics, the XGBoost model is selected for further analytical pursuits.

Visualization of feature importance

To intuitively interpret the chosen variables, the SHAP values were utilized to show what effect these elements had on the 30-day mortality in the model. In general, the more important the SHAP value of a feature is, the more influence it has on the model. We then ranked the importance of features in the best-performing XGBoost model.

In the modeling section, 25 variables were initially analyzed, out of which the XGBoost model ultimately selected 18 for inclusion in the model. Figure 4 displays these chosen variables, sorted by their importance. This ranking illustrates the extent to which each variable influences the model’s predictive performance, aiding in a deeper understanding of the decision-making process within the model.

Fig. 4

The importance ranking of the top 18 variables based on the mean (|SHAP value|). APSIII: Acute Physiology Score III; Mbp_max: Mean Blood Pressure maximum; PTT_min: Partial Thromboplastin Time minimum; AlP_min: Alkaline Phosphatase minimum; RDW_max: Red Cell Distribution Width maximum; CRRT: Continuous Renal Replacement Therapy

Figure 5 displays the 18 predictors evaluated by the mean SHAP value. The feature ranks (Y-axis) refer to the significance of each feature for the prediction model, and the SHAP values (X-axis) correspond to the impact of each feature on each sample model. The relationship between the size of the characteristic value and the predicted impact can be seen through the color, and the distribution of the characteristic value is displayed (blue indicates the high-risk value, while yellow indicates the low-risk value).

Fig. 5

The importance ranking of the 18 risk variables with stability and explanatory properties performed by the optimal model. The higher the SHAP value of the feature, the higher the risk of death for the patient. The blue portion of the feature’s value represents higher values

As depicted in Fig. 5, the risk factors for 30-day all-cause mortality are as follows: higher APSIII scores, older age, and an increased Charlson comorbidity index. Past medical history factors include peripheral vascular disease, rheumatic diseases, and a history of malignancy. Within the first 24 h in the ICU, crucial indicators include lower average oxygen saturation, higher blood glucose levels, lower body temperature, and lower mean arterial pressure. Blood tests within this time frame show longer PTT durations, increased maximum red cell distribution width, and larger anion gaps which are significant markers. Additionally, the use of vasoactive medications and Continuous Renal Replacement Therapy (CRRT) are highlighted as treatment factors.

Classification and risk stratification of predictive scores

XGBoost model was applied for predicting and stratifying the likelihood of 30-day all-cause mortality in SAP patients of the training set. All subjects in the training set were categorized into high-risk and low-risk groups, taking the maximum proximity entry index to be the best cutoff (0.62).

Survival Analysis

As shown in the Kaplan-Meier curves, the 30-day survival rate for patients identified as high-risk by the XGBoost model decreases over time, suggesting that these individuals are more likely to succumb. This observation is statistically significant (logarithmic rank test: p < 0.0001, Fig. 6).

Fig. 6

Kaplan-Meier curves for low and high ML risk groups

COX regression analysis

The correlation between high ML risk and 30-day mortality in SAP patients remained after adjustment for the first 9 most impactful variables. (adjusted HR:10.61; 95% CI:5.47–20.60; p < 0.001). Multivariable COX regression analysis is shown in Fig. 7.

Fig. 7

Multivariable Cox regression for 30-day all-cause death

Subgroup analysis

Then, subgroup analysis is performed to further verify the predictive value of the model (Table 4). The subgroup analysis revealed that the model effectively distinguished between high-risk and low-risk patients with severe acute pancreatitis, irrespective of gender, age, history of continuous renal replacement therapy, or cancer. However, it was unable to predict the 30-day mortality risk for patients with severe acute pancreatitis who also suffered from peripheral vascular disease. To further evaluate the robustness of the results, we tested cross-interactions between high- and low-risk groups and age, gender, CRRT, peripheral vascular disease, or malignant cancer. In the high- and low-risk groups, there were no interactions found between age, gender, CRRT, peripheral vascular disease, or malignant cancer.

Table 4 Subgroup analysis of diverse populations (Low-risk group as reference)

Discussion

In this research, we developed and tested an interpretable ML-based risk stratification tool for predicting the risk of all-cause mortality in SAP patients during a 30-day follow-up period. In this study, we applied 6 ML methods to construct the scoring system, among which XGBoost showed the best performance. The average AUC of this risk score is 0.881, sensitivity of 0.5714, specificity of 0.9651 and F1 score of 0.64which is significantly better than other currently available risk scores. Although ML models are often unable to output intrinsic explanations, we solve the problem of ML model interpretation by applying a state-of-the-art technique called an interpretable ML tool called SHAP. SHAP can help us to identify the first six most important feature variables of SAP mortality use of vasopressor, high Charlson comorbidity index, low blood oxygen saturation, history of malignant tumor, hyperglycemia and high APSIII score.

Compared with laboratory tests, predictive scores or models in previous studies, the risk stratification tool built with XGBoost as the main algorithm in this study can more accurately predict the 30-day mortality of acute severe pancreatitis. In previous studies, as shown in Supplementary Table 2, the AUC values for predicting 28-day mortality of SAP patients, including WBC, PLR, NLR, RDW, CRP, Bedside index of acute pancreatitis severity (BISAP), CTSI and APACHE II, were 0.796, 0.697, 0.749, 0.722, 0.595, 0.812, 0.84 and 0.78, both significantly lower than the performance of the scoring model established in this study [27,28,29]. In addition, these scoring systems have different limitations. For example, the APACHE II score is mainly used for critically ill patients rather than for AP patients and requires invasive tests such as blood gas tests [9]. Ranson scores measure 48 h of data to predict prognosis, leading to delays in patient risk management [30]; Although the CTSI score may provide essential information on the diagnosis of AP, the availability of instruments may limit the application of the score and neglect the evaluation of clinical signs and symptoms [12]. The Harmless acute pancreatitis score (HAPS) was designed to identify mild acute pancreatitis [11].

Recently, several researchers have built ML models to predict the severity of AP patients and to identify SAP patients early [31, 32]. BalazsKui et al. used decision trees, random forest, logistic regression, SVM, CatBoost and XGBoost to construct ML models to identify the severity of AP patients at an early stage. The XGBoost classifier had the strongest predictive power, with an average AUC of about 0.81. In addition, Anjuli K Luthra et al. compared the ability of GBM ML and multivariable logistic regression to predict mortality in patients with acute biliary pancreatitis. the GBM ML model had higher PPV (47.3% vs. 35.9%) and lower sensitivity (40.1% vs. 46.7%) compared with the GBM ML model multivariable logistic regression, respectively [33]. This study was aimed at SAP patients in the ICU. Six ML methods, including XGBoost and LR, were used to establish models for predicting patient mortality within 30 days. The model constructed by XGBoost had excellent prediction performance, with an AUC value as high as 0.881, sensitivity of 0.5714, and specificity of 0.9651. The PPV is 0.7273. Therefore, this model is of great value for the death prognosis of patients with severe pancreatitis and is of great significance for the patient risk management of clinicians.

Although many studies have proven the predictive power of clinical elements on the negative outcome of pancreatitis, the present study further identifies significant predictors of all-cause mortality in AP patients. Previous studies have shown that clinical characteristics, demographic characteristics, and treatment status are important bases for patient risk assessment. Consistent with previous literature and clinical experience, the first six key variables involved in this model, including the use of vasopressor, high Charlson comorbidity index, low blood oxygen saturation, history of malignant tumor, hyperglycemia and high APSIII score, are important in predicting the poor prognosis of SAP patients. For example, AP Patients treated with vasopressors had a higher risk of death compared to non-users during the follow-up period [34]. Patients with a high Charlson Comorbidity Index had higher mortality [35]. Lower SPO2 is correlated with higher fatality in AP patients [36]; The mortality rate of AP patients with diabetes was markedly higher than that of AP patients without diabetes (1.7%). AP may be the first symptom of pancreatic cancer, and patients with malignant tumors such as pancreatic cancer are more likely to have a poor prognosis [37]. The CRRT plays an important role in the treatment of SAP patients [38]. Lower mean blood pressure and higher BUN were independent risk variables for mortality in SAP patients [39]. Older SAP patients are three times more likely to die than younger patients [40]; Hypothermia can lead to worse outcomes in SAP patients [41]; In addition, longer partial thromboplastin time (PTT) [42], higher red cell distribution width (RDW) [43], alkaline phosphatase (ALP) [44], rheumatic disease [45], peripheral vascular disease [46], higher total bilirubin [47], metastatic solid tumor [48], myocardial infarct [49], SBP [39], and less WBC [50] were closely related to the high mortality of SAP patients. These results suggest that these variables are effective predictors of mortality in SAP and can prospectively provide a basis for clinicians’ risk management of SAP patients.

The research model has the following advantages: First, compared with traditional logistic regression and linear regression, the ML model uses high-order nonlinear interaction, and its prediction performance is better and more stable [17, 51]. In this study, six kinds of ML models such as XGBoost were used to build prediction models and the best ones were selected for research. Second, the black-box nature of ML algorithms limits the interpretability of predictive models, while the AI tool SHAP identifies key variables and quantifies the impact of individual features on the ultimate prediction [24, 52]. Therefore, this study uses SHAP values to explain the critical variables contained in the predictive model to help clinical practitioners understand and apply the model and contribute to patient risk management. Finally, the key features involved in this model are objective data, which avoids the subjective bias of physicians. At the same time, some limitations existed in this research. First of all, the present study only included patients from one hospital, which may cause some bias. We will further expand the scope of the study to cover patients from various areas and hospitals to optimize the performance of this model. Second, we focused only on common ML approaches to modeling and did not evaluate the performance of these models against currently used risk models. Third, based on the advantages of deep learning for building medical models, we will try to build prognostic models of AP through deep learning and conduct in-depth studies combining more comprehensive data and patient information to improve prediction.

Conclusion

In the explanatory machine learning model, the best prognostic model of severe acute pancreatitis was the XGBoost model, with an average AUC value of 0.881 ± 0.033. The first six most important characteristic variables were the use of vasopressor, high Charlson comorbidity index, low blood oxygen saturation, history of malignant tumor, hyperglycemia and high APSIII score. Based on a machine learning algorithm, the model excavates the key clinical indicators that affect the prognosis of critically ill patients and shows excellent predictive ability and risk stratification potential through comprehensive evaluation, which provides data support for treatment strategies.