GeoHeight-Bench: A New Breakthrough in Enabling Remote Sensing Large Models to "See Height"

The research team has launched GeoHeight-Bench, the first evaluation framework focused on height-aware remote sensing understanding, along with the baseline model GeoHeightChat. This addresses the "vertical blind spot" issue where existing large models ignore vertical dimension information in the remote sensing field, laying the foundation for the transition of remote sensing AI from 2D visual perception to 3D understanding.

Background: The "Flatness" Dilemma of Remote Sensing AI

When we talk about large language models and multimodal AI, we often focus on natural language processing and general visual understanding. However, in the field of Earth observation, existing large multimodal models (LMMs) almost completely ignore the vertical dimension information of "height". In real-world remote sensing applications (such as urban planning and disaster response), 3D structures are more decision-relevant than 2D textures, yet existing models perform poorly due to "flat" training.

Core Challenge: Data Scarcity and Vertical Blind Spots

The primary obstacle to height perception is the extreme scarcity of annotated data. Professional geodetic measurement equipment is required, leading to a lack of remote sensing datasets with precise height annotations. The research team refers to this as the "vertical blind spot"—models can see 2D features but cannot understand vertical relationships, which is particularly critical in disaster scenarios (such as flood prediction) and complex geometric analysis.

To address the data scarcity dilemma, the research team proposes a scalable data generation pipeline based on Visual-Language Models (VLMs). The core lies in systematic prompt engineering and metadata extraction techniques to automatically generate training data with height annotations: first, use VLMs to extract information such as ground object types and positional relationships; then, use prompt strategies to guide the generation of height-related descriptions and annotations; finally, integrate with geographic metadata to form structured samples, eliminating reliance on manual annotations.

Two Benchmarks

Based on the data generation pipeline, two complementary benchmarks are constructed:

• GeoHeight-Bench: Focuses on relative height analysis, such as judging the height relationship of ground objects or the direction of terrain slope;
• GeoHeightChat+: Requires holistic terrain perception reasoning, such as flood inundation area prediction or landmark visibility judgment.

GeoHeightChat Model

The first height-aware remote sensing large model baseline, GeoHeightChat, was developed. Its core is the collaborative fusion of visual semantics and implicitly injected height geometric features. The model receives remote sensing images while obtaining height geometric features (extracted from DEM or generated via geometric reasoning), and uses a special architecture to enable deep interaction between features. Experimental results show that this design significantly improves performance on height-related tasks, verifying the necessity of height perception.

This study systematically proposes the direction of "height-aware remote sensing understanding" for the first time, providing a complete technical path from data generation to model design and evaluation standards. Application prospects include:

• Disaster response: More accurately assess the scope and severity of disaster impacts;
• Urban planning: Intelligently analyze building density and skyline changes;
• Agricultural monitoring: Optimize irrigation and crop management by combining terrain;
• Autonomous driving: Provide more precise 3D environment understanding.

The release of GeoHeight-Bench and GeoHeightChat marks an important transition of remote sensing AI from "2D visual perception" to "3D understanding". Through innovative data generation methods and model architectures, the team has solved the vertical blind spot problem, laying the foundation for building an Earth observation system with spatial intelligence. In the future, remote sensing AI will be able to "understand" the 3D structure of the Earth, providing stronger technical support for addressing challenges such as climate change and urban development.