JAIDE: A Large Language Model Implemented from Scratch Using Zig Language, Exploring Reversible Computing and Quantum Relational Reasoning

JAIDE is a large language model framework released by kollarsandor on GitHub (2026-05-27), built from scratch using the Zig language. Its core highlights include: an innovative Reversible Scatter Flow (RSF) architecture enabling O(1) memory backpropagation, Morpheme-Guided Tokenization (MGT), the Neural Symbolic Quantum Relational (NSIR) reasoning mechanism, and hardware-agnostic execution capabilities supporting CPU, multi-GPU, and FPGA. It aims to address memory bottlenecks, computational efficiency, and hardware adaptability issues in current large model training.

Project Background

Current large model training faces core pain points such as memory bottlenecks (backpropagation requires storing a large number of activation values), low computational efficiency, and poor hardware adaptability. JAIDE attempts to rethink neural network computing from the underlying paradigm to solve these issues.

Choice of Zig Language

As a systems-level language, Zig emphasizes explicit memory management, compile-time computation, and zero-cost abstractions. Compared to Rust's ownership system, Zig provides more direct memory control; compared to C++, it has a more modern type system and compile-time reflection capabilities. Its C interoperability facilitates integration with high-performance computing libraries like CUDA, making it suitable for deep learning scenarios with extremely high performance and memory efficiency requirements.

Reversible Scatter Flow (RSF) Architecture

The core innovation of JAIDE is the bijective Reversible Scatter Flow (RSF) architecture, where each layer transformation is reversibly designed. Traditional neural networks lose intermediate states during forward propagation, requiring a large number of activation values to be stored for backpropagation; however, RSF reconstructs intermediate states through reverse computation, eliminating the need to save activation values.

O(1) Memory Backpropagation

Based on the reversible nature of RSF, JAIDE achieves constant-level memory backpropagation. The memory requirement of traditional Transformers grows linearly with sequence length and model depth, while JAIDE's O(1) design keeps memory usage relatively stable, facilitating deployment on edge devices and training large-scale models in resource-constrained environments.

Morpheme-Guided Tokenization (MGT)

JAIDE introduces the MGT tokenization strategy based on linguistic morphemes, which differs from purely statistical methods like BPE and WordPiece. Its advantages include: better cross-language transferability, more reasonable handling of rare words and compound words, token boundaries that align with human linguistic intuition, and it is particularly suitable for morphologically rich languages such as German and Turkish.

Neural Symbolic Quantum Relational (NSIR) Reasoning

The NSIR mechanism introduces the concept of probability amplitudes from quantum computing into graph neural networks to model complex relationships between entities. By performing probabilistic reasoning through quantum superposition and interference effects, it may have better expressive power than traditional attention mechanisms for complex logical tasks such as mathematical proof, code generation, and common sense reasoning.

SFD Optimizer

JAIDE is equipped with a custom SFD (Scatter Flow Descent) optimizer, specifically designed for the RSF architecture. It considers the gradient flow characteristics of reversible computing and may outperform Adam or SGD in terms of convergence speed and stability.

JAIDE leverages the compile-time metaprogramming capabilities of the Zig language to implement a hardware-agnostic execution layer:

• CPU: Uses SIMD instruction sets for vectorized computation;
• Multi-GPU: Supports cross-device parallelism and data-parallel training;
• FPGA: Generates computation kernels optimized for specific FPGA architectures.

This flexibility allows JAIDE models to be seamlessly migrated between cloud servers and edge devices without rewriting core code, making it suitable for AI applications in multiple deployment environments.

JAIDE represents an attempt to reflect on and reconstruct the current deep learning technology stack, and the directions it explores have important technical significance:

1. Memory Efficiency: Reversible computing provides a possible solution to the problem of memory scarcity in large models;
2. Hardware Diversity: The hardware-agnostic design adapts to the hardware ecosystem beyond NVIDIA GPUs;
3. Language Ecosystem: Explores a balance between the performance of systems languages and the ease of use of Python;
4. Tokenization Strategy: Verifies the value of linguistic knowledge for breakthroughs in tokenization.

As an early-stage project, the actual effect and engineering maturity of JAIDE remain to be verified, but the integrated redesign from the underlying architecture to high-level algorithms may be a necessary attempt to break through the current bottlenecks in AI computing.