Bottlenecks in Vision-Language Navigation: How 3D Scene Understanding Ability Affects Zero-Shot VLN Performance

This paper quantifies the actual impact of 3D scene understanding ability on the performance of zero-shot Vision-Language Navigation (VLN), revealing the phenomenon of perceptual saturation—when perceptual precision exceeds a threshold, the gain in navigation success rate from further improvement decreases sharply. The study proposes that 3D understanding in VLN should shift from pixel-level precision to navigation-relevant core semantics and bounding box proportions, providing new ideas for designing more efficient navigation systems.

Zero-shot VLN has attracted attention due to its low data collection cost and generalization ability. It usually integrates pre-trained Vision-Language Models (VLM) and Large Language Models (LLM): VLM constructs 3D scene graphs, while LLM handles high-level reasoning and decision-making. However, current 3D perception models prioritize pixel-level precision, which conflicts with the computational constraints and real-time efficiency requirements of navigation, becoming a key bottleneck.

Existing 3D models pursue pixel-level precision for general tasks, but in navigation scenarios, there are problems of high computational overhead, insufficient real-time performance, and information redundancy. The study decomposes the VLM-LLM navigation system into two components: 1. Slow LLM planner (relies on topological semantics for path planning); 2. Fast reactive navigator (uses spatial coordinates and bounding boxes to execute decisions).

By evaluating advanced 3D scene understanding models, the phenomenon of perceptual saturation was found—after perceptual precision exceeds a threshold, the gain in navigation success rate decreases. The study proposes a statistical success rate (SR) upper bound for the two subsystems: the planner's performance is limited by the semantic integrity of the scene, and the navigator's performance is limited by spatial positioning accuracy.

Based on the findings, the paper suggests that 3D understanding in VLN should: 1. Prioritize navigation-relevant core vocabulary (key objects, topological structures); 2. Emphasize bounding box proportions (relative positions are more important than absolute pixel precision); 3. Balance precision and efficiency (customize perception models for navigation tasks).

Evaluation using advanced 3D models validated the perceptual saturation phenomenon and the upper bound of success rate: after exceeding the precision threshold, the improvement in navigation success rate is limited; semantic understanding and spatial relationship accuracy are more critical; optimizing the navigation relevance of perception models can significantly improve system efficiency.

The study's impact on the VLN field includes: 1. Model design: Encouraging the development of navigation-customized 3D perception models; 2. Evaluation criteria: Suggesting the introduction of navigation efficiency-related metrics; 3. System architecture: Supporting layered architectures that separate high-precision perception and fast response.