MORPHEUS: A New Multimodal Pre-training Paradigm for Cancer Biology

This article introduces the MORPHEUS project, the first multimodal pre-training strategy specifically designed for cancer biology, which achieves unified representation learning of histopathology and molecular profile data through mask modeling. Keywords: Multimodal learning, cancer biology, histopathology, omics data, mask modeling, precision medicine.

As the second leading cause of death globally, the complexity of cancer research and diagnosis has long been a major challenge in the medical field. Traditional cancer research methods rely on single-modal data (histopathological images or molecular-level information), but cancer is a multi-dimensional complex disease, and a single perspective is insufficient to capture the full picture. In recent years, AI has shown great potential in medical imaging and omics data analysis, but most models are designed for a single data type. How to integrate the visual features of pathological images with the biological information of molecular profiles has become a key problem in computational oncology.

The MORPHEUS project proposes the first multimodal pre-training strategy tailored specifically for cancer biology. Drawing on the mask modeling ideas from natural language processing and computer vision fields, it innovatively applies them to unified representation learning of cancer multi-omics data. The project name is derived from the Greek god of dreams, symbolizing the "reconstruction" of a complete biological picture from multiple data sources. Its core mechanism is to mask part of the input data and train the model to infer and reconstruct the hidden parts from the remaining information.

Masked Multi-omics Modeling: During the pre-training phase, part of the molecular profile data such as RNA expression, DNA methylation (DNAm), and copy number variation (CNV) is randomly masked. Histopathological images (WSI) are used to assist in reconstructing the masked omics features, reflecting the association between the morphological information of pathological images and molecular changes. UNI-based Pathological Feature Extraction: The UNI (a large-scale self-supervised pre-training model for pathological images) is used to extract pathological features. However, due to UNI's license restrictions, pre-trained weights cannot be provided, but detailed reproduction guidelines are available. Flexible Adaptation to Downstream Tasks: The pre-trained encoder can be adapted to various downstream tasks such as survival analysis, cancer subtype classification, few-shot learning, and omics reconstruction.

MORPHEUS provides detailed data preprocessing guidelines: Pathological Image Data needs to download the pre-extracted pathological image patch features (converted into embedding vectors) of UNIv2 from HuggingFace; Molecular Omics Data is obtained from public databases (RNA expression from USCS Xena, DNA methylation and CNV from GDC Data Portal), referencing the preprocessing pipelines of the MultiSurv and DRIM projects.

Omics Reconstruction for Precision Medicine: Reconstructing molecular profiles from pathological images provides a cost-effective alternative to expensive or technically unfeasible molecular tests. Few-shot Cancer Subtype Classification: Achieves accurate classification in scenarios where labeled data for rare cancers or new subtypes is scarce. Multimodal Fusion Survival Prediction: Combines pathological image and molecular profile information to build more accurate survival prediction models, improving the prognosis evaluation system.

Current Limitations: Pre-trained weights cannot be provided due to UNI's license restrictions; downloading and preprocessing biological data has a threshold for non-professional users; multimodal pre-training requires a large amount of computing resources. Future Outlook: Expand to more omics modalities (proteomics, metabolomics); integrate clinical information (medical history, treatment plans); develop lightweight inference versions for clinical deployment; establish multi-center validation to evaluate generalization ability.