FSM-LLM: Introducing the Structural Beauty of Finite State Machines to Conversational AI

Explore how the FSM-LLM framework combines the language understanding capabilities of large language models with the structured control of finite state machines to build predictable, testable, and scalable conversational AI systems.

• Original Author/Maintainer: Nikolas Markou
• Source Platform: GitHub
• Original Title: fsm_llm: A Finite State Machine hybrid with Large Language Models
• Original Link: https://github.com/NikolasMarkou/fsm_llm
• Publication Date: 2026-05-29

---

Large Language Models (LLMs) exhibit amazing capabilities in text generation, but they are inherently stateless. Each call is independent; the model does not automatically remember previous conversation content nor follow preset business processes. This 'freedom' may be an advantage in open-ended creation, but it becomes an insurmountable obstacle in practical application scenarios that require multi-turn interactions, state tracking, and process control.

Imagine customer service robots, order processing systems, or medical consultation assistants — these scenarios all require AI to remember information provided by users, advance the conversation according to specific processes, and make consistent decisions at key nodes. Simple LLM calls are difficult to meet these needs, while traditional rule engines lack the flexibility of language understanding.

The FSM-LLM framework was born to solve this contradiction.

---

FSM-LLM adopts an elegant hybrid architecture that combines the advantages of two technologies:

LLM undertakes the core responsibilities of natural language processing in the framework:

• Intent Understanding: Parse user input and extract key entities and intents
• Data Extraction: Identify and structure important information from conversations
• Response Generation: Generate natural and coherent responses based on the current state

FSM provides the skeleton and rules for the conversation:

• State Definition: Clarify the goals and behavioral norms of each stage
• Transition Rules: Determine the conversation flow based on conditional judgments
• Context Management: Maintain cross-turn conversation states

This division of labor allows the system to retain the language flexibility of LLMs while gaining the determinism and predictability of traditional state machines.

---

The core innovation of FSM-LLM lies in its unique 'two-stage' processing flow:

When user input arrives, the system first executes:

1. Data Extraction: The LLM analyzes the input, extracts key information, and updates the context
2. Transition Evaluation: Determine whether a state transition is needed based on JsonLogic rules or LLM classification
3. State Switch: Execute state transition if conditions are met