Survival Prediction by Integrating Multimodal Pathological Images and Genomics

Integrating data from pathology images and genomics enhances cancer patient survival prediction accuracy, offering solid foundation for personalized medicine and precision treatment. To enhance predictive accuracy, we propose an intermediate fusion-based method to explore the latent relationships between histopathological images and genomic data across global and local levels. Whole Slide Image (WSI) features are extracted using ResNet50 and genomic features are extracted by Self-Normalizing Networks (SNN). Similarity measures are used to learn enable global semantic similarities across modalities. A bidirectional cross-attention module identifies dense local connections. Optimal transport methods capture global structural consistency between modalities. These features are aggregated through a Transformer encoder and gated attention pooling (GAP) to form bag-level representations. The model estimates the hazard function to predict cancer survival risk. Experimental results on the BLCA, LUAD, and UCEC WSI datasets demonstrate that the proposed method surpasses other comparative methods, effectively integrating pathology images and genomic data to significantly improve survival prediction accuracy.