FSUNav: A General Zero-Shot Navigation Architecture Inspired by the Division of Labor Between the Human Cerebrum and Cerebellum

Inspired by the collaborative division of labor between the human cerebral cortex and cerebellum, FSUNav proposes a robot navigation architecture with a "Cerebrum-Cerebellum" dual-module design. It achieves cross-platform, zero-shot, multi-modal input-based goal-oriented navigation and has reached leading levels in multiple benchmark tests. It addresses core problems of traditional navigation algorithms including heterogeneous platform compatibility, real-time and safety trade-off, insufficient open vocabulary generalization, and limited multi-modal support.

The core challenge in the field of robot navigation is autonomous goal navigation in unfamiliar environments. Traditional vision-language navigation methods have four major bottlenecks:

1. Heterogeneous platform compatibility issue: Algorithms are optimized for specific robots, leading to poor cross-platform transferability;
2. Real-time and safety trade-off: Complex models cause decision delays, making collisions easy in dynamic environments;
3. Insufficient open vocabulary semantic generalization: Dependence on predefined category IDs makes it impossible to understand descriptions of unseen objects;
4. Limited multi-modal input support: Only handles a single modality, lacking flexible interaction capabilities.

Cerebellum Module: High-Frequency End-to-End Local Planner

• A general navigation strategy trained based on deep reinforcement learning, achieving millisecond-level response speed and cross-platform generality;
• Abstracts robot states into general geometric/kinematic features, adapting to humanoid, quadruped, and wheeled robots;
• Reward design includes safety indicators such as collision avoidance and path smoothness, naturally having low collision risk;
• Receives sensor data at high frequency (10-30Hz), outputs low-level motion commands, and responds quickly to environmental changes.

Cerebrum Module: Three-Layer Reasoning and Zero-Shot Object Detection

• Semantic Understanding Layer: Uses a Vision-Language Model (VLM) to parse instructions, extract target features and spatial constraints, supporting open vocabulary;
• Scene Perception and Target Localization Layer: VLM generates candidate regions, and multi-frame verification confirms the target, reducing false detection rates;
• Global Planning and Task Management Layer: Generates a coarse-grained global path, coordinates the cerebellum for execution, and handles abnormal situations (occlusion, path blockage).

FSUNav natively supports multiple input methods:

1. Pure text description: e.g., "Find the blue vase in the living room";
2. Detailed target description: e.g., "A ceramic teacup with patterns placed on a wooden coffee table";
3. Reference image: Directly provide a target image, and the system finds similar objects;
4. Combined input: Text + image, e.g., "Find a red chair similar to the one in the image".

Users can interact naturally without learning specific formats or pre-registering targets.

• Simulated testing: On benchmarks such as MP3D, HM3D, and OVON, object navigation, instance image navigation, and task navigation all achieved state-of-the-art (SOTA) performance, with excellent results in success rate, navigation efficiency (path length/number of steps), and safety (number of collisions);
• Real-world deployment: Successfully completed complex tasks on wheeled, quadruped, and humanoid robots, verifying robustness and practical value.

Technical Significance

• Theoretical level: Demonstrates the effectiveness of bionic nervous system architecture in solving complex AI problems, providing a framework for multi-module collaborative optimization;
• Practical level: Cross-platform generality reduces algorithm adaptation costs for enterprises, zero-shot capability reduces data preparation for deployment, and multi-modal support improves user experience.

Future Outlook

• Subdivide the cerebrum module into sub-modules such as semantic understanding, spatial reasoning, and human-computer interaction;
• Fine-tune the cerebellum module to adapt to specific robot dynamics;
• Deeply integrate with Large Language Models (LLMs) to achieve complex instruction understanding and multi-turn dialogue navigation.