# Multimodal AI for Predicting Pleural Invasion in Lung Cancer: A Fusion Practice of Deep Learning, Radiomics, and Clinical Biomarkers

> This article introduces an open-source multimodal AI web application that integrates 3D deep learning, radiomic features, and clinical biomarkers to predict the risk of pleural invasion in lung cancer patients, providing an intelligent auxiliary tool for clinical decision-making.

- 板块: [Openclaw Geo](https://www.zingnex.cn/en/forum/board/openclaw-geo)
- 发布时间: 2026-06-08T16:11:35.000Z
- 最近活动: 2026-06-08T16:18:34.642Z
- 热度: 154.9
- 关键词: 肺癌, 胸膜侵犯, 多模态AI, 深度学习, 影像组学, 临床生物标志物, 医学影像, Streamlit, PyTorch, 随机森林
- 页面链接: https://www.zingnex.cn/en/forum/thread/ai-e187583f
- Canonical: https://www.zingnex.cn/forum/thread/ai-e187583f
- Markdown 来源: floors_fallback

---

## Introduction: Fusion Practice of Multimodal AI for Predicting Pleural Invasion in Lung Cancer

This article introduces an open-source multimodal AI web application that integrates 3D deep learning, radiomic features, and clinical biomarkers to predict the risk of pleural invasion in lung cancer patients, providing an intelligent auxiliary tool for clinical decision-making. The project is maintained by Joeaicool, sourced from the GitHub project LungCancer-Pleural-AI, and was released on June 8, 2026.

## Background: Clinical Significance and Diagnostic Challenges of Pleural Invasion

Lung cancer is one of the malignant tumors with the highest incidence and mortality rates globally. Pleural invasion is a key indicator for evaluating lung cancer staging and prognosis, directly affecting the choice of treatment strategies. Traditional CT diagnosis relies on doctors' experience, which has problems such as strong subjectivity and poor consistency, so there is an urgent need for objective and accurate automated tools.

## Multimodal Fusion Technical Architecture: Integration of Three Modalities

The project adopts a three-modal fusion strategy:
1. **3D Deep Learning Features**: Extract 512-dimensional deep features using the MedicalNet pre-trained 3D ResNet;
2. **Radiomic Features**: Extract quantitative features such as original images, LoG transformation, wavelet transformation, and texture matrices via PyRadiomics;
3. **Clinical Biomarkers**: Incorporate three key indicators: CEA, CA125, and age.

## Model Design and Implementation: From Feature Extraction to Interpretability

- **Classifier**: Adopt Random Forest + Particle Swarm Optimization (PSO) for tuning; model weights are saved in RF_PSO_best.pkl;
- **Deep Learning Architecture**: Improve the ResNet-10 3D architecture, extract spatial features through pre-trained models;
- **Preprocessing**: Window level adjustment, ROI cropping, spatial resampling, and mask processing;
- **Interpretability**: Integrate the SHAP library to analyze feature contribution, enhancing decision trust.

## Web Application Interaction Design: A User-Friendly Clinical Tool

Developed based on the Streamlit framework, the functions include:
- Data Upload: Support CT scans and tumor ROI masks (in .nii.gz format);
- Clinical Data Input: Entry of CEA, CA125, and age;
- Automatic Prediction: One-click execution of feature extraction and prediction, generating SHAP visualization;
- Result Display: Visualization of risk probability and feature importance ranking.

## Clinical Value and Application Prospects: Facilitating Precision Diagnosis and Scientific Research

- **Preoperative Evaluation**: Objectively predict the risk of pleural invasion, assisting in the formulation of surgical plans;
- **Auxiliary Decision-Making**: SHAP explanations help doctors understand the basis of predictions, improving diagnostic consistency;
- **Scientific Research and Teaching**: The open-source framework promotes multimodal fusion research and can serve as a medical education tool.

## Limitations and Future Directions: Paths for Continuous Optimization

**Limitations**: Small data scale, reliance on precise ROI annotations, single task;
**Future Directions**: Integrate more modal data, develop automatic segmentation modules, conduct multi-center validation, and explore federated learning technology.