Guidance: A Structured Control Language Built for Large Language Models

Guidance is a domain-specific programming language developed by Microsoft Research, designed to address the pain point where traditional prompt engineering struggles to precisely control the output structure of large language models (LLMs). It enables structured generation control, multimodal interaction, stream processing, and other capabilities through a declarative syntax, allowing developers to obtain reliable and predictable LLM outputs. It is suitable for production scenarios that require parsing and structured outputs.

As LLM capabilities improve, developers rely on them to complete complex tasks. However, the free-text outputs generated by traditional prompt engineering are difficult to control in terms of structure and format—for example, when generating JSON, they often deviate from expectations or include extra explanations, leading to parsing difficulties. Guidance was created to solve this problem, providing a declarative language to precisely control the generation process.

Guidance is not just a simple prompt template library; it is a complete language system that allows developers to programmatically define the structure and constraints of generated content. It treats generation as a controlled, composable process, supporting programming elements like variables and conditional branches. At the same time, it preserves the model's capabilities in creative generation scenarios, balancing control and flexibility.

1. Structured generation control: Enforces the model to follow specified formats (e.g., JSON/XML) through underlying mechanisms, restricts token selection at key positions, and improves output reliability;
2. Multimodal interaction: Natively supports mixed processing of text, images, etc., enabling seamless integration into multimodal applications;
3. Stream processing and real-time interaction: Supports processing while generating, allows pausing and intervention during generation, suitable for real-time scenarios;
4. Compatibility with mainstream models: Supports mainstream models like GPT, Claude, Llama, etc., and can be used without changing the underlying model.

1. Structured data extraction: For example, extracting information from resumes and generating valid JSON, ensuring correct formatting during the generation phase;
2. Multi-step reasoning and tool calling: Precisely controls the output format of each step, helping to build reliable agent systems;
3. Dialogue systems and role-playing: Constrains the model to maintain character style, avoiding off-topic or inappropriate responses.

• Compared to prompt templates: Provides procedural control rather than just templates;
• Compared to LangChain: More focused on generation control rather than full-process coverage;
• Compared to JSON Schema: Applies constraints during generation rather than post-hoc validation, improving efficiency.

Guidance represents a new paradigm for LLM generation control, elevating prompt engineering to the level of procedural control and providing a reliable tool for LLM applications in production environments. As LLM capabilities grow, the demand for generation control will become more important, and Guidance's value will become increasingly prominent—it is worth developers' in-depth exploration.