SauerkrautLM: How a 1.3M-Parameter ModernBERT Model Beats Large Language Models in Controlling Doom Agents Using Hash Embeddings

The SauerkrautLM-Doom-MultiVec project challenges the traditional assumption that "bigger models equal better performance". Based on a ModernBERT model with only 1.3 million parameters, it uses innovative hash embedding technology to outperform much larger traditional large language models in Doom game agent control tasks, providing new ideas for efficient AI model design.

Large Language Models (LLMs) have achieved remarkable results in the NLP field, but they face issues like high inference costs, large latency, and high energy consumption—these drawbacks are particularly prominent in real-time response scenarios (e.g., game control). As a classic FPS game, Doom requires agents to have fast decision-making, spatial awareness, and strategic planning capabilities, which traditional LLMs struggle to meet in terms of real-time performance.

The core of SauerkrautLM is hash embedding technology: it maps vocabulary to a fixed-size shared vector pool using hash functions, with multiple words sharing vectors. This achieves improved parameter efficiency, collision utilization (learning word similarity), and computational acceleration. Combined with the ModernBERT architecture optimized for modern hardware, it creates a compact model with only 1.3 million parameters (far fewer than the 4 million+ parameters of traditional BERT).

The model processes multimodal inputs (visual information from game screens, current status like health/ammunition, historical action sequences). It uses the MultiVec method to encode different inputs into a unified vector, which is then processed by ModernBERT to output the next action decision (movement, shooting, weapon switching, etc.). The extremely small parameter count allows inference to be completed in milliseconds, meeting the real-time requirements of game control.

Experimental results show that SauerkrautLM not only outperforms traditional models of the same size in Doom control tasks but even surpasses large language models using standard embeddings. This proves that parameter count is not the only determinant of performance—architecture design and embedding methods are equally important. Hash embeddings have great potential, opening up new possibilities for edge computing and real-time AI applications (e.g., mobile devices, embedded systems).

This project prompts reflections on AI design philosophy: elaborate design and targeted optimization are more effective than mere scale expansion, aligning with the directions of "green AI" and "small data". Future application prospects include game AI (smarter NPCs, lower hardware thresholds), robot control, autonomous driving, etc. Hash embedding technology can be extended to recommendation systems, knowledge graphs, and other fields, indicating a shift in AI from "bigger" to "smarter".