# LithoBench: Evaluation of Multimodal Large Models' Capabilities in Remote Sensing Petrology Interpretation

> This article introduces the LithoBench benchmark, which assesses the geological semantic understanding capabilities of large vision-language models in remote sensing petrology interpretation tasks. The benchmark includes 10,000 expert-annotated samples covering five cognitive levels, and experiments reveal that existing models have significant limitations in high-order reasoning tasks.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-08T12:07:26.000Z
- 最近活动: 2026-05-11T03:22:14.727Z
- 热度: 76.8
- 关键词: 遥感岩石学, 多模态模型, 基准测试, 地质语义理解, 视觉语言模型, 知识密集型任务, 专家评估
- 页面链接: https://www.zingnex.cn/en/forum/thread/lithobench
- Canonical: https://www.zingnex.cn/forum/thread/lithobench
- Markdown 来源: floors_fallback

---

## [Introduction] LithoBench Benchmark: Evaluating Multimodal Large Models' Remote Sensing Petrology Interpretation Capabilities

This article introduces the LithoBench benchmark, which assesses the geological semantic understanding capabilities of large vision-language models in remote sensing petrology interpretation tasks. The benchmark includes 10,000 expert-annotated samples covering five cognitive levels, and experiments reveal that existing models have significant limitations in high-order reasoning tasks.

## Background: Challenges in Remote Sensing Petrology Interpretation and Opportunities for Large Models

Remote sensing petrology interpretation is a fundamental task in geological surveys, mineral exploration, and regional geological mapping. It is a highly knowledge-intensive task where experts need to integrate multiple clues such as visual and spectral information to infer rock types. Traditional methods can only handle simple classification and face challenges like large intra-class differences and small inter-class differences. In recent years, multimodal large models have brought opportunities, but there is a lack of benchmarks to comprehensively evaluate their geological semantic understanding capabilities. Existing general benchmarks are too simple and lack expert-level evaluation standards.

## LithoBench Benchmark Design: Multi-level Cognitive Architecture

LithoBench is a multi-level benchmark specifically designed to evaluate geological semantic understanding in remote sensing petrology interpretation. Its features include: Scale and diversity (10,000 expert-annotated instances, 12 representative rock categories); Comprehensive task types (4,000 multiple-choice questions + 6,000 open-ended questions); Five-level cognitive architecture (Identification and description, Comparative analysis, Mechanism explanation, Practical application, Comprehensive reasoning), which can accurately locate the boundary of model capabilities.

## Data Construction: Expert-participated Semi-automated Process

To ensure data validity, an expert-participated, knowledge-driven semi-automated construction process is adopted: Structured geological image description (professional geologists annotate key information such as rock type and texture); Multi-round quality review (at least two experts review independently, and a third arbitrator is introduced for disagreements); Difficulty grading (assign cognitive levels based on expert evaluation).

## Experimental Findings: Capability Limitations of Existing Models

Evaluation results of mainstream large vision-language models on LithoBench show: High-order reasoning is still a shortcoming (basic identification is acceptable, but mechanism explanation, application, and comprehensive reasoning are insufficient); Insufficient knowledge integration capability (poor performance in handling complex problems with multi-source information); Open-ended questions are more challenging (require correct answers + reasonable explanations).

## Significance and Outlook: Promoting Geological Semantic Understanding Research

LithoBench provides a standardized evaluation platform to help objectively assess the capabilities of existing models and point out development directions. It reveals the significant limitations of current multimodal models in handling deep domain knowledge tasks, suggesting that attention should be paid to deep knowledge integration and reasoning capability improvement. In the future, it is expected to become an important tool to promote geological semantic understanding research and help develop multimodal AI systems with expert-level geological knowledge.