Reading
Vision-and-Language Navigation (VLN) agents driven by multimodal large models often face issues like route deviation and early stopping. Three-Step Nav proposes a three-step protocol of "Look Ahead - Look Now - Look Back", achieving state-of-the-art zero-shot performance without fine-tuning.
Section 01
Multimodal large model-driven Vision-and-Language Navigation (VLN) agents often encounter problems such as route deviation and early stopping. Three-Step Nav proposes a three-step protocol of "Look Ahead - Look Now - Look Back", achieving optimal zero-shot performance without fine-tuning, effectively addressing the core pain points of existing VLN agents.
Section 02
Vision-and-Language Navigation is a challenging task in the field of embodied intelligence, requiring collaboration between multiple abilities like language understanding and visual perception. Multimodal Large Language Models (MLLMs) have brought new hope to VLN, but current zero-shot VLN agents have three major issues: easy route deviation, early stopping, and low success rates. The root cause lies in MLLMs' lack of targeted modeling for navigation-specific risks (such as cumulative drift and target confusion).
Section 03
Most existing VLN methods adopt a short-sighted "single-frame decision" strategy, which struggles to handle long-range dependencies. Inspired by human navigation (global planning → local alignment → drift correction), Three-Step Nav formalizes it into a three-step protocol of "Look Ahead - Look Now - Look Back" to solve long-range decision-making problems.
Section 04
Section 05
Three-Step Nav does not require fine-tuning or gradient updates and can be plug-and-play integrated into existing VLN pipelines. It taps into the potential of off-the-shelf MLLMs through prompt engineering and structured reasoning, avoiding expensive data annotation and model training costs, and adapting to diverse VLN task scenarios.
Section 06
Three-Step Nav was evaluated on the R2R-CE (English) and RxR-CE (multilingual) continuous environment datasets. Metrics like success rate, SPL (Success weighted by Path Length), and navigation error comprehensively outperformed previous zero-shot methods, with some metrics approaching supervised learning methods, verifying the effectiveness of the three-step protocol.
Section 07
Insights: Pure prompt engineering + structured reasoning can significantly improve the performance of large models on specific tasks; the three-step paradigm can be extended to long-range decision-making tasks (e.g., robot manipulation). Future directions: Online learning for adaptive landmark extraction, explicit spatial memory modules, multi-agent collaboration. The open-source code (https://github.com/ZoeyZheng0/3-step-Nav) lowers the threshold for implementation.
Keep going with more reads from the same topic.
Splinter is a minimalist, high-performance key-value (KV) and vector storage system enabling zero-latency inter-process communication via shared memory and atomic operations. With only 766 lines of core code, it supports millions of operations per second and 768-dimensional vector storage, offering a new architectural approach for local LLM inference and data-intensive applications.
Folkering OS is the world's first AI-native bare-metal operating system, entirely written in Rust no_std without relying on Linux, POSIX, or libc. It can generate commands from scratch, compile them into WASM, and run them in 10 seconds, achieving true self-evolution.
This project explores how to use Large Language Models (LLMs) to detect logical vulnerabilities in Ethereum smart contracts, especially business logic flaws that are difficult for traditional static analysis tools to capture, such as economic manipulation and execution order errors.
This article introduces an open-source project for building modern large language models from scratch. The project evolves from a GPT-2-style basic architecture to a Llama 2/3-style production-level implementation, covering complete tutorial implementations of key technologies such as RMSNorm, RoPE, SwiGLU, GQA, and MoE.