Cogni-ML: A High-Performance Machine Learning Library Based on Crystal Language with Native Apple Silicon GPU Acceleration

Cogni-ML is a complete machine learning toolchain built from scratch based on the Crystal language. Its core feature is native support for Apple Silicon's Metal GPU acceleration, enabling local execution of quantized large language models (LLMs). Combining Crystal's C-like performance with Ruby-style elegant syntax, it provides tensor operations, automatic differentiation, neural network layers, and optimizers, filling the gap in the Crystal ecosystem for machine learning.

• Original Author/Maintainer: Sergey Kuznetsov (@skuznetsov)
• Source: GitHub (https://github.com/skuznetsov/cogni-ml)
• Release Date: 2026-06-06

The Crystal language is known for its high performance (close to C) and concise syntax (similar to Ruby). Cogni-ML is not a wrapper of existing frameworks but a complete toolchain built from scratch, aiming to provide developers with an efficient and easy-to-use ML development environment.

Basic Computing Layer

• Tensor: Generic multi-dimensional array supporting multiple numerical types
• Shape: Strongly typed dimension representation, catching shape errors at compile time
• MetalBuffer: Memory management abstraction for Apple Silicon GPU
• Autograd: Backpropagation automatic differentiation system based on computation graph

Neural Network Modules

Implements components like Linear, LayerNorm, MultiHeadAttention, ViT, all supporting Metal acceleration

Optimizers

Provides Adam (Adaptive Moment Estimation) and AdamW (Adam variant with decoupled weight decay), supporting state management

Native Metal Inference Pipeline

• Qwen 3.5 Support: Compatible with GGUF quantization formats like Q4_K/Q5_K, supporting GQA, RoPE, KV caching, chunked prefill, and wave scheduling
• Speculative Decoding: Uses Qwen 3.5 0.8B Q8_0 as the draft model, combining N-gram caching and line-batch validation to improve throughput

GGUF Format Support

Complete parser that can read metadata, load tokenizers, perform dequantization calculations, and is compatible with models from the Hugging Face/LM Studio ecosystem

• CPU Fallback: Compile with -Dcpu_only to support non-Metal environments like Linux/CUDA hosts, for model validation and testing
• Experimental CUDA Support: Implemented CUDA Driver API bindings, basic kernel testing, and quantized matrix multiplication validation, laying the foundation for a complete CUDA inference path

1. Local Embedding Generation: Native Metal acceleration for the nomic-embed-text-v2-moe model, ensuring privacy and high performance
2. Local Text Generation: Qwen 3.5 9B model can run Chinese and English generation on Apple Silicon devices like M1 Pro/Max
3. Model Research: Lightweight experimental platform with automatic differentiation and modular design, facilitating attempts at new architectures

Cogni-ML is a milestone project in the Crystal ecosystem, proving the language's feasibility for computationally intensive tasks and providing the ML community with a lightweight, high-performance local inference option. In the future, it will improve CUDA backend and multi-GPU support, and is expected to become an important choice for local LLM deployment.