FSM-LLM: Injecting Structured Thinking into Large Language Models with Finite State Machines

FSM-LLM is an open-source framework designed to address the obstacles in complex multi-turn dialogues caused by the stateless nature of large language models (LLMs). It combines the structured control capabilities of finite state machines (FSMs) with the language understanding abilities of LLMs to build predictable, testable, and maintainable intelligent dialogue systems. The core idea is to separate state control from language processing: LLMs handle language tasks, FSMs manage process control, and the framework maintains the state.

LLMs excel at natural language generation, but they are inherently stateless—each call is independent, so they cannot automatically remember dialogue context or follow predefined business processes. This makes it difficult to build complex multi-turn dialogue systems where AI assistants can both understand natural language and advance according to established processes while maintaining consistent context. FSM-LLM was created to resolve this contradiction.

Design Philosophy: Separate state and language processing—LLMs are responsible for understanding input, extracting information, and generating responses; FSMs define dialogue phases, transition rules, and the order of business logic; the framework maintains context and evaluates transition conditions.

Two-Stage Architecture:

1. Data Extraction and State Evaluation: Call the LLM to extract intent and key information to update the context, then use JsonLogic rules to evaluate whether to switch states (supports simple matching or complex LLM decisions).
2. Response Generation: Generate a response after state transition is completed, ensuring the response is based on the final state and avoiding the "outdated response" problem (e.g., the AI still asks questions after the user says goodbye).

Extended Ecosystem:

• Intent Classification: Supports single/multi-intent and hierarchical classification; automatically uses LLM-driven structured classification and provides confidence scores.
• Reasoning Engine: 9 structured reasoning strategies (analysis, deduction, induction, etc.), each implemented as an FSM for traceability and debugging.
• Workflow Orchestration: 11 step types (API calls, conditional judgments, parallel execution, etc.), supporting asynchronous event-driven workflows.
• Agent Patterns: 12 classic agent patterns (ReAct, Plan-Execute, etc.), supporting tool calls and human-machine collaboration.
• Monitoring Panel: Web-based real-time monitoring of FSM, agent, and workflow states.

Technical Highlights: Supports over 100 LLM providers (via litellm); FSM nesting and stacking; built-in security mechanisms (sensitive information filtering, etc.); 8 processor hook points for customizing behavior.

FSM-LLM is suitable for scenarios requiring structured processes:

• Customer Service Robots: Follow standard service processes (greeting → diagnosis → solution → conclusion).
• Medical Consultation Systems: Collect information strictly according to medical procedures without missing symptoms or confirmation steps.
• Financial Service Assistants: Sensitive operations require multiple confirmations and identity verification to ensure security steps are not bypassed.
• Education and Training Dialogues: Guide learning according to outlines and dynamically adjust content difficulty.

Using FSM-LLM is simple and efficient:

1. Define a JSON-formatted FSM configuration (e.g., GreetingBot, including initial state, purpose of each state, extraction/response instructions, and transition rules).
2. Run the code:
   • Python: Import the API, load the configuration from a file, start the dialogue and interact.
   • Command Line: Set the API key, run the fsm-llm command with the specified configuration file. (Refer to the original document for example configurations and code.)

FSM-LLM combines the deterministic skeleton of FSMs with the flexible flesh of LLMs, representing a pragmatic approach to AI application development. It provides a predictable, auditable, and maintainable solution for enterprise-level dialogue systems, allowing developers to retain the language capabilities of LLMs while gaining the structural advantages of traditional software engineering. As AI moves from experimentation to production, this "controlled intelligence" will become an important choice for enterprises.