From Inference Routing to Agent Orchestration: A Declarative Policy Compilation Framework with Cross-Layer Validation

This paper proposes a non-Turing-complete declarative policy language (Semantic Router DSL), extending the single-request routing of LLM inference gateways to multi-step agent workflow orchestration. By compiling a single source file into multi-target outputs, it achieves unified governance with traceable auditing, controllable costs, and verifiable policies, aiming to address the pain point of fragmented policy governance in LLM production deployment.

In LLM production deployment, policy governance faces fragmentation issues: inference teams, security teams, infrastructure teams, and agent teams maintain policy rules in different systems and formats respectively. When business changes occur, multi-party coordination is required to ensure consistency, leading to low efficiency and easy policy drift.

Design Philosophy

Adopting a non-Turing-complete design, limiting the language's expressive power in exchange for stronger analyzability and verifiability.

Core Components

• Content Signals: Input sources include embedding similarity, PII detection, jailbreak score, etc.;
• Weighted Projection: Weighted calculation of signals to form decision-making basis;
• Priority Decision Tree: Tree structure organizing policies, supporting conditional branches and priority sorting;
• Structured Audit Trail: Complete recording of decision-making processes, forming traceable logs.

Advantages of Single-File Governance

Policies are centralized in a single declarative source file, supporting version control, code review, and traceable changes.

Capability Expansion

Initially used for model selection in inference gateways, now extended to multi-step agent workflow orchestration, with policy decisions running through the entire execution process.

Multi-Target Compilation Outputs

• Orchestration Frameworks: LangGraph node-edge definitions, OpenClaw agent configurations;
• Kubernetes Infrastructure: NetworkPolicy, Sandbox CRD, ConfigMap;
• Network Device Configurations: YANG/NETCONF data models;
• Protocol Boundary Gateways: MCP, A2A protocol gateways.

This capability ensures that policy changes synchronously update all related systems, eliminating the risk of policy drift.

Auditability

Non-Turing completeness allows exhaustive analysis of decision paths. The compiler generates a complete decision tree, and audit logs are closely coupled with logic, making decisions traceable.

Cost Efficiency

Intelligent routing assigns simple requests to lightweight models and complex requests to large models, reducing inference costs; centralized management reduces redundant development and maintenance overhead.

Verifiability

Guarantees during compilation: exhaustive routing (no undefined behavior), no conflicting branches, and reference integrity.

Tunability

Policy parameters (thresholds, weights) are adjusted centrally, and a single compilation propagates to all target systems. For example, adjusting the PII detection threshold can be automatically applied to multiple layers.

Validation Boundaries at Each Layer

• Policy Layer: The compiler statically verifies DSL syntax, logical conflicts, and missing references;
• Orchestration Layer: Target frameworks (LangGraph/OpenClaw) verify configuration correctness in the test environment;
• Infrastructure Layer: Policy-as-Code tools in the CI/CD pipeline verify K8s configurations;
• Runtime: Integration tests and observability tools monitor actual behavior.

Clear boundary division helps establish a layered quality assurance system.

Policy-as-Code Maturity

Incorporate policies into software engineering practices, using version control, code review, and automated testing to improve policy quality.

Value of Non-Turing-Complete Languages

Using restricted languages in specific domains in exchange for stronger analyzability and verifiability, complementing general-purpose languages.

End-to-End Consistency First

Through multi-target compilation of a single source file, ensure consistency across the entire chain rather than local optimality.

Limitations

The non-Turing-complete design limits the expressive power of complex policies, requiring a balance between expressiveness and verifiability.

Future Directions

• Explore dynamic policy adjustment (based on runtime feedback) while maintaining verifiability;
• Research integration with reinforcement learning to allow the system to learn optimal policy parameters from data while maintaining auditability.