Small Models Can Win Too: Spatial Reasoning Experiments on a 16GB MacBook Reveal the Boundaries of LLM Capabilities

An experiment conducted on a regular 16GB MacBook shows that the smallest 1B-parameter model outperformed larger models in specific spatial reasoning tasks, challenging the traditional assumption that "the larger the model, the stronger its capabilities". The study tested four open-source small models, revealing that the relationship between model size and specific capabilities is not a simple positive correlation, and that monitoring mechanisms cannot rescue capabilities the model itself does not possess. The research is open-source and cost-free, providing a new perspective for LLM evaluation.

The AI field has long assumed a positive correlation between model size and capabilities, but this study found that small models outperformed larger ones in specific spatial reasoning tasks. The study's subtitle, "Monitoring Cannot Rescue What a Model Cannot Produce", points out the core insight: if a model lacks a certain capability, monitoring mechanisms cannot create that capability out of thin air. This finding re-examines the boundaries of model capabilities and the effectiveness of safety monitoring.

The experiment selected three programmatic spatial reasoning tasks: folding reasoning (testing spatial imagination), maze navigation (testing path planning); the participating models are four open-source small models: Qwen2.5-1.5B, Qwen2.5-3B, Llama-3.2-1B, Llama-3.2-3B, all of which can run locally on a 16GB MacBook.

The experiment results show no model won all tasks:

Model	Folding Task1	Folding Task2	Maze Task
Qwen2.5-1.5B	55%	0%	34%
Qwen2.5-3B	10%	0%	0%
Llama-3.2-1B	5%	10%	54%
Llama-3.2-3B	15%	20%	30%
Qwen2.5-1.5B performed best in Folding Task1, and Llama-3.2-1B performed best in the Maze Task, both outperforming larger models, confirming that capabilities match task characteristics rather than having a positive correlation with size.

The study adopted a "validator-guided rejection sampling" strategy (K=64), attempting up to 64 generations for each question, with a deterministic physical validator selecting the best answer. The validator can accurately calculate folded shapes or confirm maze paths, avoiding black-box issues and reflecting the trend of leveraging existing model capabilities.

The study proves that consumer-grade hardware (16GB MacBook) can complete meaningful AI research (cost 0, time 14 hours); model selection should not blindly pursue size—small models have cost-effectiveness advantages in specific tasks (low inference cost, flexible deployment, privacy protection); open-source code and data facilitate reproduction and expansion.

Limitations: Limited sample size (folding n=20, maze n=50), insufficient statistical confidence. Future directions: Introduce a third model family for validation, test GPT-4-level large models as a control, conduct prompt sensitivity research to exclude the impact of prompt engineering.

This study reminds us: model capabilities are multi-dimensional, and a single indicator cannot fully evaluate them; small models have unexpected advantages in specific fields; monitoring mechanisms have inherent limits; research on consumer-grade hardware still has value. The future of AI needs to balance size and efficiency, focusing on task specificity and resource utilization.