MultiSeismo: Multimodal AI Enters the Field of Seismology

Title: MultiSeismo: Multimodal AI Enters the Field of Seismology Abstract: The first multimodal dataset and dedicated model for seismology are released, integrating waveform data, geographic images, and text descriptions, opening a new path for cross-modal understanding in the scientific field. Original Author/Source: arXiv authors, Source Platform: arxiv, Original Title: MULTISEISMO: A Multimodal Seismic Dataset and Model for Cross-Modal Seismic Understanding, Original Link: http://arxiv.org/abs/2605.26320v1, Release Date: 2026-05-25. Core Content: This study fills the gap in multimodal data integration in seismology, promoting the application of multimodal AI in seismic science through datasets, instruction sets, and dedicated models.

General multimodal models have limitations in professional scientific fields, rooted in the lack of diverse scientific data types (such as time series, sensor readings, etc.) in training data. Seismology requires integrating multi-source information like waveform records, epicenter locations, and geological structures, but existing seismic datasets are either single-modal or lack standardized multimodal integration, limiting the application potential of multimodal AI.

1. MultiSeismo Dataset: The first large-scale structured multimodal seismic dataset, covering 16000+ seismic events from 2010 to 2023, integrating waveform data, intensity distribution maps, population exposure maps, and text descriptions to support cross-modal learning.
2. MISCE Instruction Set: A multimodal instruction dataset built from raw data, converting data into a supervised learning format, supporting tasks from basic retrieval to complex cross-modal analysis (e.g., magnitude judgment, impact assessment, etc.).
3. SeisModal Model: Based on the Unified-IO 2 architecture, adding a dedicated time-series encoder to process waveform data and achieve domain adaptation.

Evaluations show that general multimodal models experience a significant performance drop when processing seismic waveform time series, while SeisModal, through domain adaptation, demonstrates significant advantages in multimodal reasoning tasks, being able to more accurately understand waveform features, associate spatial information, and generate reasonable explanations.

Cross-Modal Understanding Challenges:

• Time-space association: Establishing mappings between waveform temporal features and geospatial distributions
• Multi-scale reasoning: Integrating millisecond-level waveform details with multi-level information of hundreds of kilometers of geological structures
• Uncertainty quantification: Identifying and expressing uncertainties in seismology

Scientific AI Insights:

• High-quality multimodal datasets are the foundation
• Domain-specific architecture adaptation is necessary
• Instruction dataset construction requires participation of domain experts

The research team plans to expand the dataset's geographic and temporal coverage, include data on submarine earthquakes, induced earthquakes, etc., and develop more complex cross-modal tasks; this methodology can be extended to other data-intensive scientific fields such as climate science and astrophysics.

MultiSeismo is an important step for multimodal AI to penetrate professional scientific fields, proving that through well-designed datasets and architecture adaptation, general models can be transformed into effective scientific tools. For the seismology community, it provides stronger AI-assisted tools; for the AI field, it offers a case of professional solutions, filling the gap in AI for seismology.