llm_sim: An Observable Simulator for Internal Behaviors of Large Language Models

llm_sim is a Python project designed specifically for educational purposes. It simulates the complete reasoning process of large language models through a modular architecture (including prompt construction, tokenization, reasoning agent, tool calling, and token-by-token generation) and provides JSON execution trace visualization. Its core value lies in transparently revealing the black-box reasoning process of LLMs, making it suitable for teaching, debugging understanding, or architecture learning.

The reasoning process of real large language models (LLMs) is usually a black box, making it difficult to intuitively understand their internal mechanisms. llm_sim does not aim for real model performance; instead, it focuses on clarity and observability. By explicitly recording every intermediate step into JSON trace files, it allows users to deeply explore the model's 'thinking process' and helps learners understand the working principles of LLMs.

It adopts a modular architecture where each component only depends on the Trace data class, ensuring strong maintainability and extensibility. Core modules include: trace.py (execution trace recording), prompt_builder.py (prompt template combination), tokenizer.py (dynamic vocabulary tokenization), llm_core.py (token-by-token generation logic), tools.py (calculator and knowledge base tools), agent.py (reasoning layer), and pipeline.py (top-level orchestration). Key design decisions: component isolation, unified interface, dynamic vocabulary, and full traceability.

1. Prompt Construction: Wrap system prompts and user inputs into labeled structured templates; 2. Tokenization: Split text using regular expressions and map to IDs via dynamic vocabulary; 3. Reasoning Agent: Detect intent through heuristic rules (call calculator for mathematical computation, call knowledge base for factual queries); 4. Token-by-token Generation: Candidate sampling, scoring (including repetition penalty and target enhancement), temperature-scaled softmax, and recording of complete candidate lists; 5. Result Assembly: Combine tool outputs and generated text into the final answer.

Three interaction methods are provided: 1. Web Interface: Animate pipeline stages, color-code token generation probabilities, and support viewing full traces; 2. Trace Viewer: Collapsible step cards, JSON syntax highlighting, inline probability bar charts, and reasoning trace visualization; 3. CLI Tool: Scripted use for quick testing, outputting JSON trace files and final answers.

Session isolation mechanism ensures multi-user data security: each user's trace is stored in an independent directory, and audit logs (audit.jsonl) record all operations and are not accessible via browsers. Deployment methods: local development (venv + server.py), production environment (Gunicorn), and Docker containerized deployment.

Applicable to multiple scenarios: 1. Teaching Demonstration: Show LLM internal principles without complex real models; 2. Architecture Learning: Understand modular design through source code; 3. Debugging Understanding: Observe intermediate steps to understand why answers are generated; 4. Prototype Verification: Quickly verify new architecture ideas; 5. Security Research: Understand the security boundaries of tool calls and knowledge retrieval.

llm_sim is a carefully designed teaching tool. With its transparent architecture and visualization capabilities, it makes LLM internal mechanisms accessible. Although it does not pursue real performance, it excels in observability and educational value. It is an excellent learning resource for developers, researchers, and students, and its modular design can serve as a basic framework to extend more complex simulation functions.