Language Geometry and Agent Systems: A Technical Note Repository on LLM Reasoning

Introducing the oldnordic.github.io project, a collection of technical notes on language geometry, LLM reasoning, and agent AI systems. It provides researchers with a unique theoretical perspective, aiming to decode the internal black-box mechanism of LLMs from a geometric viewpoint. Maintained by oldnordic, the project was published on June 11, 2026, with the original link: https://github.com/oldnordic/oldnordic.github.io.

Large Language Models (LLMs) have transformed AI understanding, yet their internal working mechanisms remain a black box. Researchers have begun to explore their essence from a mathematical and geometric perspective, attempting to establish a rigorous theoretical foundation. Against this backdrop, the oldnordic.github.io project compiles relevant technical notes to offer a theoretical perspective.

Language geometry is an emerging field that maps natural language to geometric spaces and uses mathematical tools to analyze semantic structures and reasoning. Core assumptions: Lexical concepts are represented in high-dimensional vector spaces; semantic relationships correspond to geometric relationships; reasoning is a spatial transformation or path. Differences from word vectors: It focuses more on structured representations (phrases, sentences, discourses), reasoning modeling (geometric operations), and interpretability (geometric intuition).

The project explores the geometric interpretation of the Transformer architecture: 1. The attention mechanism is a selective projection in the semantic space; 2. The feedforward network is a nonlinear geometric transformation; 3. Inter-layer propagation is a trajectory in high-dimensional space. Directions for reasoning path visualization: Use t-SNE/UMAP dimensionality reduction to project to 2D/3D, track internal state changes in complex reasoning tasks, and identify key turning points or decision boundaries.

Traditional LLMs are static, while agent systems have memory, tool use, and planning capabilities. From a geometric perspective: 1. The state space consists of current states (memory, environmental observations); 2. Tasks correspond to target manifolds/regions; 3. Actions are movements in the state space; 4. Planning is path search. This framework can explain emergent behaviors: Complex behaviors emerge when there is sufficient freedom; tool use opens new channels; self-reflection is meta-level trajectory analysis.

The notes cover the following topics: 1. Foundations of language geometry (semantic space structure, vector geometric properties, similarity measurement); 2. LLM reasoning mechanisms (internal geometry of Transformers, few-shot learning theory, geometric perspective of chain-of-thought); 3. Agent architectures (memory modeling, tool calling and state transition, spatial model of multi-agent collaboration); 4. Practical experiments (visualization tools, case studies, open-source implementations). Research value: Theoretically, it improves interpretability, provides a unified framework, and guides design; practically, it aids model debugging, security analysis, and efficiency optimization.

Language geometry faces the following challenges: 1. Curse of dimensionality (the representation space of real LLMs has extremely high dimensions, making geometric intuition ineffective); 2. Nonlinear complexity (nonlinearity of neural networks makes precise analysis difficult); 3. Lack of formalization (more heuristic observations, fewer strict mathematical theories); 4. High computational cost (geometric analysis and visualization require significant resources).

oldnordic.github.io represents the trend of AI research returning from engineering practice to theoretical foundations, reminding us of the importance of deeply understanding the essence of systems. The language geometry perspective provides a valuable thinking framework and is a precious learning resource for researchers and engineers. As AI systems become more complex, interdisciplinary exploration (mathematics, physics, cognitive science, etc.) will bring breakthrough progress.