# ExTrm: A Journey of Exploring Experimental Miniature Reasoning Models Built with Elixir

> ExTrm is an experimental reasoning model project based on the Elixir language, exploring two architectural directions: grid reasoning and text/code reasoning, providing a lightweight experimental platform for research-oriented AI development.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-18T23:56:37.000Z
- 最近活动: 2026-05-19T00:23:42.829Z
- 热度: 159.6
- 关键词: Elixir, 推理模型, Nx, ARC, 网格推理, 代码生成, 实验性AI, 函数式编程
- 页面链接: https://www.zingnex.cn/en/forum/thread/extrm-elixir
- Canonical: https://www.zingnex.cn/forum/thread/extrm-elixir
- Markdown 来源: floors_fallback

---

## ExTrm: Exploring Experimental Miniature Reasoning Models Based on Elixir (Introduction)

ExTrm is an experimental miniature reasoning model project entirely written in Elixir, aiming to provide a lightweight experimental platform for research-oriented AI development. It explores two core directions: grid reasoning (to solve ARC-style tasks) and text/code reasoning (supporting long contexts and code generation), using Nx as the tensor computation backend. The project is positioned as a research codebase rather than a production-grade framework, focusing on rapid iteration to validate architectural ideas.

## Project Background and Positioning

In the AI field, most reasoning models are built based on the Python ecosystem, but Elixir has become a new choice due to its concurrency model and functional programming features. ExTrm is not a mature production framework but a **research codebase**, aiming to provide an experimental platform for rapid iteration, trial-and-error, and validation of new architectures. Its code style is intentionally kept rough to prioritize idea validation, making it suitable for developers willing to experiment hands-on.

## Research Direction 1: Grid Reasoning Model

Grid reasoning was the initial core of the project, focusing on solving ARC (Abstraction and Reasoning Corpus) style tasks—pattern recognition and reasoning on colored grids. Key architectural elements include: recursive block structure (supporting multi-step reasoning), repeated thinking steps (simulating human reasoning), and colored grid representation. The default parameter count is about 100 million; running on CPU requires passing smaller configuration parameters.

## Research Direction 2: Text/Code Reasoning Model

The text/code reasoning direction is more practical, with components including: byte-level tokenizer (no pre-trained vocabulary needed), Hugging Face dataset integration, long-context text model foundation, and a complete training/saving/inference pipeline. The project uses the `karti06k/Qwen-59k-Python-Instruct` dataset for training, which contains instructions, reasoning processes, and code, suitable for code generation models.

## Highlights of Technical Architecture

**Long Context Support**: The text model supports a context length of 128K at the architectural level, with configuration parameters such as `context_length:131072` and `attention_window:2048`. It uses techniques like chunked pre-filling, sliding attention, RoPE-style positional encoding, and memory tokens, but its actual capabilities need to be verified through training. **Nx Backend**: Uses Nx (Numerical Elixir) as the tensor computation backend, providing a NumPy-like API while maintaining a functional style, making it easy for Elixir developers to conduct deep learning experiments.

## Quick Start Guide and Code Structure

**Code Structure**: Modules are clearly divided. The core of the grid model is in `lib/ex_trm/model.ex`, the core of the text model is in `lib/ex_trm/text/model.ex`, and there are also modules for dataset loading, training, inference, and command-line tools. **Quick Start**: 
1. Environment preparation: `mix deps.get` → `mix test`
2. Small-scale grid experiment: Need to pass small configuration parameters (e.g., `vocab_size:8`, `d_model:16`, etc.)
3. Text model operations: Download dataset (`mix ex_trm.text.dataset.download`), small-scale training (`mix ex_trm.text.train --rows64 --steps10`), inference test (`mix ex_trm.text.generate --prompt ...`)

## Target Audience and Usage Suggestions

ExTrm is suitable for: AI enthusiasts in the Elixir ecosystem, model architecture researchers (for rapid idea validation), educational purposes (learning reasoning model mechanisms), and ARC task researchers. It is not suitable for developers expecting an out-of-the-box production framework. It is recommended to treat it as a **laboratory workbench** to freely experiment with architectural ideas without pursuing a perfect API.

## Conclusion: The Democratization Direction of AI Development

ExTrm proves that AI development does not have to be limited to the Python ecosystem. Elixir's concurrency features and fault-tolerant design are theoretically suitable for distributed training. Although it is in the early experimental stage, it provides a valuable starting point for the Elixir community. As the author said: "Some ideas may stay, some may get deleted, some may become separate models. That's fine." This open experimental mindset drives the development of AI frontiers.