Lung most cancers danger prediction utilizing interpretable ensemble fashions on life-style and scientific knowledge

1. Introduction

The fashionable life-style has induced many important and life-threatening ailments in humankind all over the world. Among these, most cancers continues to be the main reason behind mortality, answerable for round 10 million deaths in 2020, which equates to at least one in each six fatalities globally [1]. Lung most cancers is likely one of the commonest kinds of most cancers [2]. According to the World Health Organization (WHO), 1.8 million individuals died worldwide in 2020 attributable to lung most cancers. Hence, it has turn out to be a major problem for healthcare professionals and sufferers. It is famous that if the illness is detected at an early stage, an individual’s possibilities of survival may be elevated.

For the early detection of lung most cancers, it’s essential to precisely distinguish between benign (non-cancerous) and malignant (cancerous) cells [3]. Various lung analysis strategies exist that utilise imaging strategies, together with CT (computed tomography), isotope, X-ray, and MRI (magnetic resonance imaging), in addition to molecular screening, amongst others [4]. These image-analysis-based diagnostic strategies nonetheless face the problem of distinguishing benign from malignant cancers, as they share similarities. Importantly, these strategies make use of a reactive method, i.e., they’ll detect a cancerous cell as soon as it has turn out to be such. Instead, a proactive method could be a greater tactic, through which the likelihood of an individual getting most cancers, primarily based on her life-style knowledge, is predicted. Machine studying strategies present a possibility to establish patterns in life-style and scientific knowledge which will assist earlier danger evaluation.

Machine studying has proven important potential for medical professionals and researchers to find hidden prospects in medical, scientific, and demographic knowledge and higher serve human beings [5,6]. In the healthcare sector, machine studying is helpful for analysing ailments with acceptable accuracy, permitting medical doctors to make higher medical choices, which in flip results in improved analysis and prognosis. Machine studying strategies are efficiently utilized to foretell and diagnose deadly ailments, similar to most cancers, coronary heart illness, and mind tumours. However, conventional machine studying approaches could battle to seize more and more complicated and non-linear relationships current in massive heterogeneous healthcare datasets [7].

Traditional machine studying algorithms, similar to logistic regression (LR), naive Bayes (NB), Ok-nearest neighbour (KNN), assist vector machine (SVM), linear discriminant evaluation (LDA), and so forth., have some limitations in most cancers prediction because of the illness’s complexity, dynamicity, and the heterogeneity of knowledge [8]. Some of the numerous limitations are summarized under.

• Limited characteristic choice: Feature choice poses a major problem in most cancers prediction utilizing typical machine studying algorithms. Cancer is a posh ailment impacted by a number of variables, together with age, gender, life-style selections, genetic mutations, and environmental influences. Identifying probably the most related options that may successfully forecast most cancers prevalence stays difficult.
• Inefficiency in dealing with imbalanced knowledge: Cancer knowledge exhibits an imbalance, the place people identified with most cancers symbolize a smaller inhabitants than these with out most cancers. Conventional machine studying algorithms could present bias towards the dominant class, resulting in suboptimal efficiency in most cancers prediction.
• Inefficiency in dealing with non-linearity: Cancer knowledge usually displays non-linear traits, indicating that feature-outcome associations usually are not linear. Conventional machine studying algorithms, similar to logistic regression and linear discriminant evaluation, assume linear associations between options and outcomes. However, when utilized to most cancers prediction duties, this assumption could produce suboptimal efficiency.
• Overfitting: Conventional machine studying algorithms are susceptible to overfitting, the place the mannequin’s extreme complexity captures noise fairly than underlying patterns. Overfitting can lead to suboptimal generalization efficiency and diminished predictive accuracy in most cancers prediction.
• Interpretability: Machine studying algorithms are sometimes characterised as black-box fashions, providing restricted transparency into mechanisms governing most cancers prediction. The significance of interpretability in scientific decision-making lies in serving to clinicians perceive components that contribute to most cancers prediction.

Employing extra subtle machine studying methodologies, similar to deep studying and ensemble strategies, could assist handle these constraints. In current years, ensemble studying paradigms, similar to boosting, bagging, voting, and stacking, have attracted rising consideration from the analysis neighborhood. These approaches have proven promising outcomes, demonstrating improved accuracy within the prediction of ailments [9].

Ensemble studying represents a machine studying paradigm that mixes the predictions of a number of particular person fashions to enhance total accuracy and scale back the chance of overfitting [10]. Several advantages are related to using ensemble studying strategies in most cancers prediction. Some of them are talked about under.

• Increased accuracy: The integration of a number of fashions by means of ensemble studying can improve most cancers prediction accuracy by leveraging particular person strengths. Each mannequin displays distinct strengths and weaknesses, and thru their mixture, the ensemble mannequin can yield larger accuracy in comparison with any single mannequin.
• Robustness: Ensemble studying strategies can improve mannequin robustness by mitigating the results of outliers and noisy knowledge. Outliers can considerably impression the efficacy of a person mannequin. However, in ensemble fashions, the affect of outliers diminishes because of the larger detection probability of different fashions throughout the ensemble.
• Better generalization: Ensemble studying strategies mitigate overfitting by combining a number of fashions with distinct biases and assumptions. The ensemble mannequin displays a decreased propensity to evolve to knowledge noise, thereby capturing basic patterns and resulting in enhanced generalization efficiency.
• Flexibility: Ensemble studying is a flexible method that may be utilized to Decision Trees (DTs), neural networks, and assist vector machines. It may be tailor-made to handle numerous most cancers prediction duties, together with analysis, prognosis, and danger stratification.

It is crucial to acknowledge that the easy software of ensemble studying doesn’t inherently yield the anticipated benefits. The number of ensemble strategies and foundational fashions needs to be knowledgeable by the particular traits of the issue, dataset properties, and out there computational assets [10]. When utilized appropriately, ensemble studying has the potential to markedly improve the efficiency of machine studying fashions [11].

Although ensemble fashions enhance predictive accuracy, they usually lack interpretability and obscure the prediction course of. Furthermore, whereas explainable synthetic intelligence (XAI) is more and more prevalent in healthcare, its software in ensemble fashions for lung most cancers prediction stays restricted [12–14]. Most analysis has targeted on elucidating particular person fashions with XAI fairly than complicated ensemble programs, thereby constraining the comprehensibility of those fashions in real-world scientific contexts. Despite developments in predictive fashions inside healthcare, there stays a paucity of fashions that combine the accuracy of ensemble studying with the interpretability of XAI for lung most cancers. Given the multitude of interconnected components influencing these circumstances, understanding mannequin predictions is crucial for clinicians to tailor remedies. Traditional ensemble strategies, though efficient, usually perform as “black boxes,” concealing the rationale behind predictions. This opacity diminishes their utility in scientific settings, the place offering clear, patient-specific insights for moral, knowledgeable decision-making is essential.

To handle this problem, this research proposes an XAI-enhanced stacking mannequin that not solely delivers excessive predictive efficiency for lung most cancers danger evaluation but additionally facilitates clear interpretation of mannequin outputs. In the area of scientific decision-making, the place transparency and accountability are important, interpretable AI strategies similar to SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) play a pivotal function. SHAP values present each international and native insights into characteristic significance, facilitating a complete understanding of the components influencing lung most cancers danger predictions. LIME, alternatively, gives localised explanations, enabling the interpretation of particular predictions on an instance-by-instance foundation. By integrating these explainability strategies, we guarantee statistical rigor in efficiency validation, thereby rendering our method extra dependable and explainable for scientific adoption.

The main contributions of this paper are as follows:

• We develop and consider a number of ensemble studying methods (boosting, bagging, voting, and stacking) for lung most cancers prediction utilizing life-style and scientific knowledge.
• We carry out a scientific comparability of base learners (LR, KNN, NB, SVM, and LDA) and show how ensemble strategies leverage their complementary strengths.
• We analyze mannequin efficiency throughout authentic, balanced, and upsampled datasets, highlighting the impression of knowledge preprocessing on predictive outcomes.
• We present that the stacking ensemble achieves probably the most constant total efficiency throughout a number of analysis metrics, whereas additionally figuring out trade-offs with respect to particular person metrics.
• We incorporate explainable AI strategies (SHAP and LIME) to offer each international and patient-specific interpretability, linking mannequin outputs to clinically significant components.
• We conduct statistical significance evaluation utilizing non-parametric assessments to validate that the noticed efficiency variations are strong and never attributable to random variation.
• We evaluate a number of ensemble methods below authentic, balanced, and upsampled knowledge settings to look at how knowledge distribution influences predictive efficiency.

The remainder of the paper is organized as follows. Section 2 presents the associated work. Section 3 describes the proposed methodology and gives an outline of the thought-about machine studying and ensemble studying algorithms. Section 4 particulars the dataset. Section 5 presents and analyzes the experimental outcomes. Sections 6 and seven report the statistical significance evaluation and interprets the proposed fashions utilizing XAI, respectively. Section 8 discusses the findings and their scientific implications. Finally, Section 9 concludes the research, outlining its limitations and instructions for future work.

2. Related work

Machine studying and ensemble-based methodologies have demonstrated substantial potential to provide constant, dependable, and legitimate outcomes, thereby discovering widespread software throughout numerous real-world domains. In the context of lung most cancers prediction, important analysis has employed these strategies to reinforce diagnostic accuracy. For instance, Abdullah et al. [15] carried out a comparative evaluation when it comes to accuracies SVM, KNN, and a convolutional neural community (CNN) in classifying lung most cancers in its early phases. In their research, SVM achieved the very best accuracy (95.56%), whereas KNN achieved the bottom (88.40%). Patra [16] utilized machine studying classifiers, together with NB, KNN, and Radial Basis Function (RBF), utilizing the Weka instrument to analyse the lung most cancers knowledge. The dataset consisted of 32 cases with 57 options, collected from the University of California, Irvine (UCI) Machine Learning Repository. Among all of the classifiers, RBF achieved the very best accuracy of 81.25% utilizing 10-fold cross-validation. The creator recommended that characteristic choice strategies and integration with different machine studying fashions might improve the efficacy of the proposed work. Radhika et al. [17] employed machine studying strategies similar to DT, SVM, NB, and LR on two publicly out there datasets (Data World and UCI) for lung most cancers detection. By using the UCI dataset, the LR mannequin attained the very best accuracy fee of 96.90%, whereas SVM outperformed nicely and achieved an accuracy of 99.20% through the use of the world knowledge repository.

Numerous research have employed quite a lot of datasets, algorithms, and methodologies to advance analysis in lung most cancers prediction. An rising variety of researchers have explored the effectiveness of ensemble studying strategies in enhancing diagnostic and prognostic accuracy for lung most cancers. For occasion, Ahmad and Mayyaa [18] designed a instrument for lung most cancers prediction utilizing bagging and randomized node optimization. The authors thought-about numerous components related to lung most cancers, similar to genetic dangers, air air pollution, smoking, mud allergy, and so forth. and signs like coughing blood, fatigue, dry cough, shortness of breath, chest ache, and others. In a associated research, Safiyari and Javidan [19] employed 5 broadly adopted ensemble strategies, together with bagging, dagging, AdaBoost (ADB), multi-boosting, and random subspace utilizing knowledge from the Surveillance, Epidemiology, and End Results (SEER) program. Wang et al. [20] launched an ensemble-based mannequin combining Random Forest (RF) with a self-paced studying bootstrap method to reinforce lung most cancers classification and prognosis. Their methodology employed a sampling technique that progressively built-in knowledge from high- to lower-quality samples. Mamun et al. [21] applied numerous ensemble algorithms, together with Extreme Gradient Boosting (XGB), Light Gradient Boosting Machine (LGBM), bagging, and ADB, evaluated by means of 10-fold cross-validation. Among these, XGB demonstrated superior efficiency, attaining an accuracy of 94.42%.

Ensemble studying methodologies, when paired with XAI, have attracted important consideration in lung most cancers prediction. This curiosity stems from their twin skill to reinforce predictive efficiency whereas offering interpretability—a vital part in scientific decision-making and early analysis. Numerous research have aimed to stability predictive accuracy with transparency. For occasion, RF has been reported to attain 98.9% accuracy throughout numerous datasets, highlighting its potential in medical diagnostics [22]. Another modern framework, DeepXplainer, combines CNNs with XGB and achieves 97.43% accuracy. This mannequin employs SHAP to supply each native and international interpretability, thereby enhancing the transparency of the system’s predictions [23]. An interpretable radiomics-based mannequin was developed utilizing a stacked interpretable sequencing cells (SISC) framework, which not solely demonstrated state-of-the-art outcomes but additionally supplied visualization instruments like response maps to help radiologists in choice assist [24]. Similarly, the ExMed platform, which includes a number of characteristic attribution strategies, has been utilized to artificial healthcare knowledge to foretell lung most cancers sufferers’ life expectancy, additional illustrating the applicability of XAI along side ensemble studying fashions [25]. In one other contribution, Siddhartha et al. [26] used a bagging-based ensemble methodology to categorise the survival standing of lung most cancers sufferers by figuring out and decoding the chance components. They developed a machine studying pipeline (RF) by mitigating the issue of imbalanced knowledge. The central contribution of this research lies in its use of XAI to interpret RF predictions, a important side for fostering belief and reliability in healthcare functions. However, the research lacks a comparative evaluation with different ensemble or deep studying fashions, which limits insights into the relative effectiveness of RF. Zhang et al. [27] aimed to develop a prediction mannequin utilizing machine studying to establish high-risk teams for early identification and prevention of lung most cancers. The efficiency of the fashions, together with LR, NB, RF, and XGB was evaluated utilizing the AUC (space below the curve), Brier loss, log loss, precision, recall, and F1 scores. The SHAP interpreter was used to visualise the fashions, displaying the affect of every predictive issue on lung most cancers danger. Another research, using the SEER 2020 database, in contrast a number of conventional classifiers—LR and DT—with ensemble strategies like XGB, LGBM, and ADB, in addition to deep studying fashions similar to DNNs. LGBM emerged because the main performer, with an accuracy of 87.45% and an AUC of 0.91 [28]. The research highlighted the necessity to combine XAI instruments to reinforce the interpretability of deep studying frameworks, notably for his or her potential use in scientific trials.

In abstract, integrating ensemble strategies—particularly superior strategies like stacking—with interpretability frameworks similar to SHAP holds important promise for enhancing each the efficiency and transparency of lung most cancers prediction fashions. These mixed approaches assist construct clinician belief in AI programs, addressing a long-standing barrier to AI deployment in scientific settings [29,30]. However, the present physique of labor primarily focuses on a restricted set of ensemble strategies, with minimal comparative analysis throughout a broader vary of fashions. Most research additionally confine themselves to traditional efficiency indicators like accuracy and F1-score, overlooking extra nuanced metrics that would higher assess scientific applicability. Furthermore, whereas XAI has been employed in lots of research, its integration with complicated ensemble frameworks stays underexplored. There is proscribed understanding of how mannequin interpretability may be systematically built-in into scientific workflows or how totally different options work together synergistically to have an effect on mannequin outputs. This hole underscores the necessity for future analysis to analyze extra complicated characteristic interactions, broader datasets, and rigorous efficiency analysis frameworks. Additionally, comparative research that consider ensemble strategies alongside conventional fashions might present invaluable insights into their respective benefits, limitations, and real-world applicability in healthcare settings.

3. Research methodology

This part gives a complete overview of the analysis methodology employed and the ensemble studying strategies utilised within the experiment.

4. Dataset description and manipulation

The lung most cancers dataset utilized on this research was sourced from a publicly accessible repository on Data World (https://www.kaggle.com/datasets/mysarahmadbhat/lung-cancer), contributed by staceyinrobert. This dataset includes 309 data and contains 16 demographic attributes. Among these, the primary 15 variables function predictors (unbiased variables), whereas the ultimate attribute represents the goal variable (dependent variable). An in depth description of the dataset attributes is supplied in Table 1. Before mannequin implementation, the dataset was examined for knowledge high quality points, particularly outliers and lacking values. The interquartile vary methodology was employed to establish potential outliers, and imputation strategies have been used to evaluate lacking knowledge. However, no outliers or lacking values have been detected, indicating a clear and full dataset appropriate for evaluation.

The authentic dataset was comparatively small for coaching ensemble studying fashions. To study the affect of dataset dimension and sophistication distribution on predictive efficiency, two augmentation methods have been adopted. First, the minority class was balanced utilizing the Synthetic Minority Over-sampling Technique (SMOTE). Second, each lessons have been upsampled to create a considerably bigger dataset for investigating mannequin behaviour below elevated pattern availability. Accordingly, three dataset variants have been thought-about:

1. i. Actual: The authentic dataset was used with none augmentation to guage mannequin efficiency below actual knowledge circumstances.
2. ii. Balanced: As proven in Fig 1, the unique dataset displays appreciable class imbalance. To mitigate this, the minority class was augmented utilizing SMOTE till each lessons contained an equal variety of samples, leading to a balanced dataset comprising 528 observations.
3. iii. Upsampled: To examine the impact of elevated pattern dimension on mannequin studying, each lessons have been synthetically enlarged, producing an upsampled dataset containing 2,387 observations.

SMOTE was employed to generate artificial minority-class samples by interpolating between neighbouring minority cases within the characteristic house. The implementation adopted the usual configuration of the imbalanced-learn library utilizing sampling_strategy = ‘auto’ and k_neighbors = 5. Under this setting, every artificial pattern was generated by interpolating between a minority occasion and one in all its 5 nearest minority neighbours recognized utilizing the Euclidean distance metric. The similar SMOTE configuration was used persistently all through all augmentation experiments to make sure methodological consistency.

After producing three datasets, normalization was utilized to every utilizing the min-max scaling method, as outlined in Eq. 1, the place, x denotes the attribute worth, and x_min and x_max symbolize the attribute’s minimal and most values, respectively. This methodology remodeled the values of all attributes to a standardized vary between 0 and 1, thereby making certain uniformity in scale throughout the datasets.

(1)

Because the unique dataset was extraordinarily small and extremely imbalanced, the balanced and upsampled datasets have been generated previous to cross-validation. We acknowledge that this method could introduce optimistic bias and doesn’t absolutely get rid of the potential of info leakage. Consequently, the outcomes obtained on the balanced and upsampled datasets needs to be interpreted primarily as comparative analyses of class-balancing methods fairly than definitive estimates of real-world efficiency.

Table 2 exhibits the descriptive statistics of the parameters throughout the three datasets. Here, complete report counts in every dataset, together with the imply, normal deviation, and minimal and most values of every attribute, are famous.

5 Experiment, outcomes, and dialogue

This part presents and discusses the experimental particulars and outcomes achieved utilizing the designed ensemble algorithms for lung most cancers prediction.

6 Statistical significance

To rigorously assess efficiency variations among the many evaluated fashions, a non-parametric framework was employed utilizing the Friedman aligned-rank check, adopted by Holm-adjusted put up hoc comparisons. The outcomes point out clear and statistically important variations throughout fashions.

As proven in Table 11, the Friedman check yields extremely important outcomes for each the voting and stacking with chosen base fashions ensembles (p ≈ 0.000), resulting in rejection of the null speculation. This confirms that the noticed efficiency variations are statistically significant fairly than attributable to random variation.

The rating leads to Table 12 additional substantiate this discovering. Both voting and stacking with chosen base fashions obtain the very best rank scores (105.86 and 107.86, respectively), clearly outperforming all baseline fashions. Among the person learners, GB and RF persistently rank because the next-best-performing fashions. In distinction, less complicated fashions similar to SVM, KNN, and Naive Bayes occupy the bottom ranks, indicating comparatively weaker efficiency.

The put up hoc evaluation gives detailed pairwise comparisons. As reported in Table 13, the voting (with chosen fashions) mannequin exhibits statistically important enhancements over a number of baseline strategies, together with SVM, KNN, NB, BME, AdaBoost, CatBoost, LDA, and BDT. However, no statistically important variations are noticed compared with stronger fashions similar to GB, RF, LGBM, LR, DT, and ET. In distinction, the STS mannequin demonstrates broader statistical superiority; it achieves statistically important enhancements over a wider set of fashions, together with XGBoost along with weaker learners. Similar to voting (with chosen fashions), its variations from top-performing fashions similar to GB and RF usually are not statistically important, indicating comparable efficiency on the highest degree.

Finally, the Wilcoxon check leads to Table 14 verify that the proposed ensemble approaches yield statistically important enhancements on the dataset degree (p ≈ 0.0425), additional supporting the robustness of the noticed efficiency positive aspects.

7 Explainability of the stacking and voting fashions

XAI encompasses methodologies that improve the transparency and comprehensibility of AI fashions, thereby rendering their choices accessible to human specialists [48]. This is especially important in domains similar to healthcare, the place belief and accountability are paramount [49]. XAI helps each international and native explanations, that are important for making certain the reliability and utility of fashions in healthcare contexts. Global explanations elucidate the first components influencing illness outcomes, thereby aiding clinicians and researchers, whereas native explanations make clear how these components have an effect on particular person sufferers, successfully bridging the hole between analysis and scientific apply.

To increase the interpretability of the proposed stacking and voting fashions for predicting lung most cancers on a worldwide scale, we employed the SHAP methodology [50]. Grounded in Shapley values from cooperative recreation concept, SHAP gives a constant framework for assessing every characteristic’s impression on particular person predictions, making it a broadly adopted instrument for elucidating complicated machine studying fashions [51]. Additionally, we utilized LIME for native or instant characteristic interpretation [52], providing fast, case-specific insights into mannequin predictions, which is especially useful for real-time decision-making.

8 Discussion and scientific implications

The improvement of interpretable ensemble fashions for lung most cancers prediction utilizing life-style and scientific indicators carries essential scientific implications, notably for early detection and danger stratification. The experimental outcomes point out that ensemble approaches—particularly stacking and voting—persistently outperform particular person base learners. While efficiency improved considerably on the balanced and upsampled datasets, these outcomes needs to be interpreted with warning, as artificial knowledge augmentation could not absolutely seize the complexity and variability of real-world scientific populations. In this context, the stacking mannequin’s efficiency on the unique dataset (93.53% accuracy) gives a extra sensible estimate of its sensible utility. The near-perfect accuracy noticed on the upsampled dataset demonstrates the mannequin’s skill to take advantage of further coaching info however shouldn’t be seen as consultant of anticipated scientific efficiency.

Although the stacking mannequin achieves the very best total efficiency throughout a number of analysis metrics, its recall is marginally decrease than that of some particular person base fashions. This highlights an inherent trade-off in classification, the place optimizing for total efficiency could result in slight reductions in sensitivity. In the context of lung most cancers prediction, recall is especially important as a result of false negatives can delay analysis and therapy. Therefore, whereas the stacking mannequin gives balanced and strong efficiency, mannequin choice in scientific settings could require prioritizing recall relying on the particular software.

To additional study mannequin behaviour, studying curve evaluation was carried out throughout totally different dataset configurations. On each the unique and SMOTE-balanced datasets, the voting and stacking fashions exhibit carefully aligned coaching and validation curves, indicating steady studying dynamics and good generalization. However, a divergence emerges within the absolutely upsampled dataset: the voting mannequin exhibits a transparent hole between coaching and validation efficiency, suggesting susceptibility to overfitting with artificial knowledge. In distinction, the stacking mannequin maintains shut alignment between the curves, demonstrating higher robustness and resistance to overfitting below the identical circumstances.

These observations are additional supported by stratified cross-validation, which helps mitigate variance in efficiency estimates. Notably, the stacking mannequin achieves persistently sturdy outcomes throughout all dataset variants, indicating that its efficiency isn’t solely depending on artificial augmentation however displays a extra steady and generalizable studying functionality.

The statistical outcomes point out that the stacking mannequin achieves probably the most constant and highest total efficiency, with important enhancements over a number of baseline strategies. However, its efficiency is statistically similar to sturdy tree-based fashions similar to GB and RF. From a scientific perspective, this distinction is essential. The statistical superiority of stacking over weaker fashions helps its use to enhance reliability, notably by lowering variability throughout affected person instances. This is efficacious in screening or triage settings, the place constant identification of high-risk people is important.

At the identical time, the dearth of statistically important variations between stacking and top-performing tree-based fashions means that comparable predictive efficiency may be achieved with much less complicated fashions. In sensible phrases, which means in settings the place interpretability, ease of deployment, or computational effectivity are priorities, fashions similar to RF or GB could function efficient alternate options. Conversely, in eventualities the place robustness throughout heterogeneous knowledge is crucial—similar to multi-center screening applications or populations with various scientific profiles—the stacking mannequin affords a bonus by integrating numerous studying patterns. This can assist keep steady efficiency when knowledge traits shift.

Through complete SHAP and LIME analyses, we recognized a number of clinically related predictors that each verify and problem the established understanding of lung most cancers danger components. A deep dive into the characteristic significance utilizing SHAP reveals an enchanting alignment with, and some shocking deviations from, established scientific data. The constant identification of fatigue (FT) and alcohol consumption (AC) as high predictors strongly helps the mannequin’s scientific validity. Fatigue is a well-documented constitutional symptom of malignancy, usually linked to systemic irritation and tumor metabolic exercise [53]. However, fatigue is a nonspecific and prevalent symptom in lots of persistent circumstances, suggesting that its predictive energy is strongest when mixed with different indicators, similar to smoking historical past or age [54]. Similarly, persistent alcohol use is a identified, dose-dependent danger issue that may promote carcinogenesis [55,56]. The mannequin’s skill to seize the dose-response relationship, through which larger values push SHAP values constructive (elevated danger), mirrors scientific observations and reinforces confidence within the mannequin’s studying course of. However, epidemiological proof on alcohol as an unbiased danger issue for lung most cancers is combined, with some research suggesting a synergistic impact with smoking attributable to enhanced carcinogen publicity (e.g., acetaldehyde) [57,58]. While smoking stays probably the most established danger issue, our mannequin attributes substantial predictive weight to alcohol consumption. This discovering warrants additional investigation—whether or not the mannequin captures true organic danger or confounding behavioural patterns (e.g., heavy drinkers being extra more likely to smoke). Prospective cohort research might make clear this relationship earlier than scientific integration. Furthermore, the mannequin’s emphasis on swallowing issue (SD) is of excessive scientific significance. Dysphagia is a major “red flag” symptom, usually indicating native tumor development the place the mass could also be compressing the esophagus, suggesting a extra superior stage [59].

However, the evaluation additionally highlights key factors for dialogue, notably relating to options that both battle with or add nuance to established danger hierarchies. One notable commentary is the comparatively low significance attributed to smoking (SK), regardless of its well-established function as the first danger issue for lung most cancers [60]. As proven within the characteristic significance analyses offered in Section 5.3.5, this sample was persistently noticed throughout each voting and stacking fashions and throughout all three dataset variants.

To additional examine this obvious discrepancy, we carried out a multicollinearity evaluation utilizing the Variance Inflation Factor (VIF). For a predictor , the VIF is computed as:

(3)

the place denotes the coefficient of dedication obtained by regressing on all remaining predictors. Larger VIF values point out stronger multicollinearity and higher redundancy amongst correlated variables.

The outcomes of VIF evaluation are offered in Table 15. Smoking (VIF = 11.16) and several other smoking-associated predictors, together with yellow fingers (VIF = 19.08), persistent cough (VIF = 17.47), shortness of breath (VIF = 18.78), and alcohol consumption (VIF = 17.79), exhibited substantial multicollinearity. This means that the predictive sign related to smoking is distributed throughout correlated behavioural and symptom-related variables. Consequently, attribution strategies similar to SHAP could allocate significance throughout these associated predictors fairly than concentrating it solely on smoking. Therefore, the comparatively decrease significance assigned to smoking shouldn’t be interpreted as contradicting established epidemiological proof, however fairly as a consequence of multicollinearity throughout the dataset.

An identical sample is noticed for different clinically established components. The comparatively low significance assigned to age (AG) and chest ache (CP) can also be counterintuitive, on condition that superior age is a significant non-modifiable danger issue for most cancers [61], and chest ache is a generally reported scientific symptom in lung most cancers sufferers [62]. These observations point out that the mannequin could also be capturing extra instant, discriminative indicators—similar to fatigue, persistent illness, and swallowing issue—over broader or much less particular indicators. Additionally, dataset-specific traits, similar to class distribution and illustration bias (e.g., aged sufferers with non-malignant respiratory circumstances), could additional affect these patterns.

Importantly, these observations spotlight a basic distinction between predictive modeling and epidemiological causality. SHAP-based characteristic significance displays the contribution of options to the mannequin’s predictions primarily based on the out there knowledge distribution, fairly than their true causal function in illness improvement. As a end result, options which are strongly correlated with the result within the dataset—or that seize downstream results—could obtain larger significance than main danger components. Therefore, deviations from established scientific hierarchies needs to be interpreted as insights into the mannequin’s discovered representations and dataset traits, fairly than contradictions of medical data.

Another intriguing high-impact characteristic is allergy (AL). The relationship between allergy symptoms and lung most cancers is complicated and never absolutely understood. Some analysis suggests {that a} hyper-vigilant immune system in allergic people would possibly supply a protecting impact in opposition to sure cancers, together with lung most cancers [63], though the connection varies relying on the particular allergy and most cancers kind [64]. The mannequin’s constant recognition of this characteristic suggests it has recognized a powerful predictive sample within the knowledge, warranting additional scientific investigation.

Other clinically related options, similar to persistent illness (CD), show notable predictive contributions, with biologically believable mechanisms involving inflammatory pathways. It is smart, as pre-existing circumstances, notably respiratory ones, are identified to extend lung most cancers danger [65]. The minimal affect of gender (GD) contradicts identified epidemiological patterns [66,67], probably reflecting altering smoking demographics or dataset limitations. These findings emphasise the significance of contextual interpretation when making use of mannequin outcomes to scientific decision-making. Future iterations might enhance these options by integrating extra detailed genetic or biomarker knowledge to extend precision.

The scientific implications of those findings might be important. The fashions would possibly enhance danger stratification by figuring out high-risk people past present smoking-based standards, probably broadening eligibility for low-dose CT screening. Including signs like fatigue and coughing could permit earlier detection than conventional strategies, whereas highlighting modifiable danger components, similar to alcohol consumption, might information preventive counselling.

The native, patient-specific explanations supplied by LIME additional improve the mannequin’s scientific utility. By displaying how a mixture of things, like excessive alcohol consumption and fatigue, contributes to a person’s high-risk rating, the mannequin gives actionable insights. This can empower physicians throughout affected person consultations, enabling them to have extra knowledgeable discussions about particular life-style modifications and the concrete components that elevate the affected person’s private danger. Ultimately, whereas the stacking mannequin demonstrates highly effective predictive capabilities, its true scientific promise is unlocked by its interpretability, which gives the essential bridge between statistical predictions and significant, actionable scientific insights.

Overall, from a scientific workflow perspective, the interpretability provided by SHAP and LIME can play a sensible function in decision-making. As illustrated in Fig 35, affected person demographic info, life-style components, and scientific signs may be processed by the proposed stacking mannequin to generate each a lung most cancers danger rating and SHAP/LIME-based explanations highlighting probably the most influential predictors. Global explanations can help clinicians in figuring out high-risk profiles primarily based on combos of life-style and scientific components, probably supporting early screening choices similar to prioritising sufferers for additional diagnostic analysis (e.g., low-dose CT scans). At the person degree, LIME-based explanations present patient-specific insights, serving to clinicians perceive why a selected affected person is flagged as excessive danger and guiding focused follow-up investigations. The ensuing danger rating and explanatory components can then be reviewed alongside typical scientific findings to assist risk-stratified choices, together with routine monitoring, further scientific evaluation, or specialist referral. Furthermore, the identification of modifiable contributors, similar to alcohol consumption, could assist personalised counselling and preventive interventions. Importantly, these explanations additionally permit clinicians to evaluate whether or not mannequin predictions are clinically believable, thereby reinforcing belief and enabling knowledgeable use of AI-assisted suggestions.

Overall, from a scientific workflow perspective, the interpretability provided by SHAP and LIME can play a sensible function in decision-making. Fig 35 illustrates a conceptual decision-support workflow meant to show how the proposed framework might be built-in right into a future scientific setting following applicable validation. Patient demographic info, life-style components, and scientific signs may be processed by the proposed stacking mannequin to generate each a lung most cancers danger rating and SHAP/LIME-based explanations highlighting probably the most influential predictors. Global explanations can help clinicians in figuring out high-risk profiles primarily based on combos of life-style and scientific components, probably informing choices relating to additional diagnostic analysis (e.g., consideration of low-dose CT screening). At the person degree, LIME-based explanations present patient-specific insights, serving to clinicians perceive why a selected affected person is flagged as excessive danger and guiding focused follow-up investigations. The ensuing danger rating and explanatory components can then be reviewed alongside typical scientific findings to assist danger stratification, together with routine monitoring, further scientific evaluation, or specialist referral. Furthermore, the identification of modifiable contributors, similar to alcohol consumption, could assist personalised counselling and preventive interventions. Importantly, these explanations additionally permit clinicians to evaluate whether or not mannequin predictions are clinically believable, thereby reinforcing belief and enabling knowledgeable use of AI-assisted suggestions. The proposed workflow is meant solely as a conceptual illustration of how explainable AI might assist scientific decision-making and shouldn’t be interpreted as a advice for scientific deployment or referral choices primarily based on the current research alone.

However, implementation challenges warrant cautious consideration, together with the subjective nature of key predictors similar to fatigue and anxiousness, which can necessitate using standardised evaluation instruments to make sure dependable scientific measurement. Although the surprising deviations don’t invalidate the prediction, they spotlight an essential implication: a clinician utilizing this instrument should interpret the output completely. The mannequin signifies the presence of a “symptom,” and the clinician should hyperlink it to the underlying “cause.” This emphasises that the instrument is meant to assist, not substitute, scientific judgment.

From a deployment perspective, there’s an inherent trade-off between predictive efficiency and computational complexity. The proposed stacking ensemble achieves the strongest predictive efficiency however requires higher computational assets than particular person base learners attributable to its multi-stage structure and meta-learning course of. While this extra complexity is unlikely to pose a major problem in most scientific environments, the place danger prediction is carried out offline and doesn’t require real-time response, it could turn out to be an essential consideration in large-scale inhabitants screening programs or resource-constrained healthcare settings. Therefore, the selection of mannequin ought to stability predictive accuracy with out there computational assets and deployment necessities.

Improving characteristic measurement with extra exact knowledge on key variables, similar to smoking (pack-years) and alcohol consumption (normal drinks), might improve predictive accuracy. Combining this with imaging outcomes and molecular markers could result in the event of multimodal prediction instruments, whereas causal evaluation can make clear whether or not the recognized associations replicate real organic relationships. These developments will likely be important for translating mannequin outputs into actionable scientific insights.

9 Conclusions, limitations, and additional scope

Lung most cancers stays one of many deadliest ailments worldwide. Early prediction and detection are important for well timed medical intervention and enhancing affected person outcomes. Machine studying strategies may be utilized for this objective. This research adopted ensemble studying to foretell lung most cancers. Four main ensemble approaches (boosting, bagging, voting, and stacking) have been designed and evaluated for lung most cancers prediction utilizing a life-style dataset. The outcomes confirmed that ensemble fashions outperformed particular person base learners, with the stacking ensemble attaining the strongest total efficiency. On the unique dataset, the stacking mannequin attained an accuracy of 93.53%, demonstrating its potential for lung most cancers danger prediction utilizing life-style and scientific indicators. Although considerably larger efficiency was noticed on the balanced and upsampled datasets; nonetheless, these outcomes needs to be interpreted within the context of artificial knowledge augmentation. Overall, the findings show that ensemble studying can successfully leverage complementary predictive patterns to enhance lung most cancers danger evaluation and assist early detection efforts. When decoding fashions utilizing XAI, in line with the SHAP methodology, options similar to allergy, alcohol consumption, fatigue, persistent illness, and peer strain for smoking and alcohol most importantly affect lung most cancers danger prediction, whereas gender, age, and shortness of breath have the least affect. The findings assist the event of focused interventions and danger administration methods to enhance affected person outcomes in lung most cancers analysis and therapy.

This research has a number of limitations that warrant cautious consideration. A main limitation is the reliance on a single publicly out there Kaggle dataset comprising 309 samples, which inherently restricts the generalizability of the findings. The mannequin’s efficiency could subsequently replicate traits which are particular to this dataset and its underlying demographic and scientific distribution fairly than broadly consultant patterns. The comparatively small pattern dimension additional will increase the chance of overfitting and studying dataset-specific relationships. Additionally, the absence of exterior validation limits the robustness of the outcomes, because the proposed fashions weren’t evaluated on unbiased datasets collected from totally different establishments or affected person populations. Consequently, the findings needs to be interpreted as preliminary, and the proposed framework shouldn’t be used to information scientific choices or low-dose CT referral till potential and exterior validation has been accomplished.

Furthermore, the dataset could not seize the total spectrum of life-style and scientific danger components related to lung most cancers. Real-world affected person populations are extremely heterogeneous, influenced by comorbidities, environmental exposures, genetic predispositions, and different components that aren’t absolutely represented within the present knowledge, probably limiting the mannequin’s applicability in sensible settings. The use of artificial knowledge augmentation introduces further limitations. The technique of upsampling each minority and majority lessons to considerably enlarge the dataset is unconventional and should distort real-world class distributions, probably resulting in overly optimistic efficiency estimates. Accordingly, the outcomes obtained on the balanced and, particularly, the upsampled datasets needs to be interpreted as comparative analyses below artificial knowledge augmentation fairly than as estimates of anticipated real-world scientific efficiency. In addition, making use of resampling previous to cross-validation could introduce bias in efficiency analysis. Specifically, utilizing SMOTE earlier than cross-validation may cause artificial samples derived from the identical authentic cases to seem throughout coaching and validation folds, introducing info leakage and probably inflating efficiency estimates.

To construct upon this work and handle these limitations, a number of enhancements may be explored. The most important subsequent step is to guage the ensemble fashions on bigger, extra numerous, and externally validated datasets. This will contain sourcing knowledge from a number of establishments to evaluate the generalizability and robustness of our method throughout totally different populations and scientific settings. Such validation must also embody a broader vary of life-style and scientific traits to create a extra complete and correct predictive mannequin. Further validation experiments might discover the mannequin’s utility in multi-cancer research. Assessing the efficiency of the proposed ensemble framework throughout different most cancers varieties would assist confirm its generalizability past lung most cancers and supply insights into its broader applicability in oncology. Methodologically, integrating our ensemble strategies with deep studying and superior characteristic engineering could additional increase predictive efficiency. Investigating novel hybrid architectures that synergistically mix these approaches represents a promising route for enhancing predictive accuracy. Ultimately, the scientific worth of this analysis lies in its sensible software. Developing a user-friendly choice assist instrument that includes ensemble fashions might facilitate the sensible software of lung most cancers danger evaluation and early detection in scientific settings. Translating these fashions right into a sensible instrument with a physician-friendly interface is a important step for profitable scientific deployment, facilitating real-time danger evaluation and supporting early detection methods for lung most cancers.

Additionally, implementing bias mitigation methods—similar to analysing underrepresented populations—will likely be essential to enhancing mannequin equity and making certain equitable healthcare outcomes. While SHAP elucidated characteristic significance, the research didn’t completely discover the interpretability of ensemble fashions. Understanding the decision-making mechanisms of those fashions is crucial for his or her implementation in healthcare, the place belief and transparency are paramount. Future analysis might examine mannequin interpretability by means of numerous frameworks to make clear relationships between enter knowledge and predictions. Incorporating multi-modal knowledge, similar to imaging knowledge with scientific indicators, might improve the predictive functionality and scientific relevance of fashions.

Future enhancements contain broadening the scope to include further base learners and hybrid fashions that combine ensemble strategies with deep studying to reinforce predictive accuracy. Examining demographic variety inside databases will likely be essential to making sure equitable healthcare outcomes. Integrating a number of knowledge sources can assist mitigate biases and enhance the mannequin’s skill to establish people in danger. The inclusion of further danger components, similar to genetic predispositions and environmental influences, in ensemble fashions warrants additional investigation. Expanding the set of assessed traits might improve their predictive energy and relevance to personalised therapy.

To progress from analysis to scientific use, potential validation throughout a number of centres is essential. Including causal inference strategies, similar to counterfactual evaluation, can assist decide whether or not important components, similar to alcohol consumption, are causally associated or merely correlated. Additionally, combining imaging and biomarker knowledge might enhance predictive accuracy, resulting in a multi-modal AI diagnostic system. Lastly, creating an interpretable decision-support instrument utilizing ensemble fashions might improve real-world usefulness in healthcare. This know-how would help healthcare professionals in assessing danger and early detection of lung most cancers, finally enhancing affected person outcomes and healthcare companies.

Acknowledgments

The authors thank the UTHealth Cancer Genomics Core for offering technical assist. The authors additionally gratefully acknowledge the assist supplied by the Deanship of Scientific Research, Vice Presidency for Graduate Studies and Scientific Research, King Faisal University, Saudi Arabia.