Latent Bridge Games: Real-Time Game Agents Connecting Fast Multimodal Models and Slow Reasoning Models

Project Core

Latent Bridge Games proposes an innovative "Latent Bridging" architecture that connects frozen fast multimodal models and slow reasoning models, resolving the "speed vs. intelligence" contradiction in real-time game AI to achieve efficient and intelligent decision-making.

Source Information

• Original Author/Maintainer: Bojie Li
• Source Platform: GitHub
• Project Link: https://github.com/bojieli/latent-bridge-games
• Release Date: June 12, 2026

When building game AI agents, developers face a fundamental contradiction:

• Fast multimodal models: Process visual/audio inputs in real time but lack deep reasoning capabilities;
• Slow reasoning models (e.g., o1, DeepSeek-R1): Can make complex decisions but have slow reasoning speeds, failing to meet real-time game requirements.

Traditional solutions require compromises between capability and speed: either sacrifice intelligence for real-time performance or accept latency for decision quality.

Dual-Model Collaboration Mechanism

The system deploys two frozen models:

1. Fast multimodal model: Perceives the game environment in real time, processes visual inputs at high frame rates, and provides instant environmental representations;
2. Slow reasoning model: Runs in the background, performing in-depth analysis and strategy planning on the latent representations from the fast model.

Latent Space Alignment

The key breakthrough is the establishment of a "Latent Bridging" mechanism: converting the output representations of the fast model into a format understandable by the slow model. Alignment occurs at the latent space level (not raw input), enabling efficient information transfer.

Representation Distillation

Train a lightweight bridge network to learn mapping the middle-layer features of the fast multimodal model to the input space of the reasoning model. Both models remain frozen, eliminating the need for expensive joint training.

Asynchronous Reasoning Pipeline

• The game main loop is driven by the fast model to ensure real-time responses;
• The slow reasoning model runs asynchronously in an independent thread, periodically receiving sequences of latent representations accumulated by the fast model to generate high-level strategic guidance.

Strategy Fusion

The final decision is a dynamic fusion of the fast model's instant response and the slow model's strategic guidance. Weights can be adaptively adjusted based on game states: prioritize speed in emergencies and decision quality at strategic moments.

This architecture has wide-ranging applicable scenarios:

• Real-Time Strategy (RTS) games: Achieve both fast micro-operations and macro strategy simultaneously;
• Competitive game AI: Demonstrate human-level reaction and superhuman strategy in fast-paced games;
• Robot control: Provide real-time perception and deep planning capabilities;
• Autonomous driving: Balance instant obstacle avoidance and long-term path planning.

Design Paradigm Insight

It demonstrates the design paradigm of "combining specialized AI systems": using architectural design to combine models with different strengths, achieving a 1+1>2 effect. This avoids the expensive path of pursuing an "all-capable single model" and uses complementary existing models to engineer solutions to complex problems.

Implications for Developers

In future AI system design, effectively combining multiple specialized models may be more cost-effective than training larger single models. The idea of "division and collaboration" is worth learning from.