Cross-attentive deep–radiomic feature fusion for multi-modal MRI-based brain tumor assessment

Abstract

Accurate diagnosis of brain tumors from multi-modal magnetic resonance imaging (MRI) remains challenging due to low image contrast, scanner variability, noise, limited annotations, and class imbalance. These factors often affect the reliability and generalizability of automated diagnostic models. To address these challenges, this study proposes a cross-attentive deep–radiomic fusion framework that hybridizes a 3D residual convolutional neural network (CNN) encoder, a Vision Transformer (ViT) branch for global contextual representation, and handcrafted radiomics features extracted from MRI data. A cross-attention fusion mechanism is introduced to effectively combine deep features with radiomic representations, enabling improved tumor segmentation and classification performance. The framework is trained using an adaptive preprocessing pipeline and enhanced with robustness strategies including adversarial training, test-time augmentation, and bias-field perturbation simulation. Experiments conducted on the combined BraTS 2019–2021 datasets demonstrate strong performance compared with recent approaches, achieving a Dice score of 0.912 for whole tumor segmentation and an AUC of 0.965 for tumor grade classification. The incorporation of radiomics features further improves diagnostic accuracy while enhancing interpretability through SHAP-based feature attribution and spatial attention visualization. The proposed approach can support AI-driven clinical decision support and health informatics systems by providing reliable and interpretable tumor assessment from multi-modal MRI data. Ablation studies confirm that each architectural component contributes to the overall performance of the model. Although the Transformer branch improves global contextual learning, it increases computational complexity and memory requirements. Future work will therefore investigate lightweight optimization strategies such as knowledge distillation and model pruning, as well as the integration of multi-parametric imaging modalities and federated learning for privacy-preserving multi-institutional analysis.