BARRED: Building Customized Policy Guardrails by Synthesizing Training Data Through Asymmetric Debate

The BARRED (Boundary Alignment Refinement through REflection and Debate) framework generates high-quality synthetic training data using dimension decomposition and multi-agent debate validation, requiring only task descriptions and a small number of unlabeled samples. It addresses the manual annotation bottleneck in building customized policy guardrails, enabling small fine-tuned models to outperform proprietary large language models in this task.

In the practical deployment of LLMs, customized policy guardrails face the following challenges:

1. Limitations of General Safety Models: Unable to capture subtle differences in vertical domains (e.g., discussions on drug side effects in medical consultations are easily misjudged);
2. Prompt Engineering Bottleneck: Inconsistent performance on boundary cases, high reasoning costs, and difficulty in scaling;
3. Supervised Learning Annotation Bottleneck: High-quality annotations in professional fields are expensive and time-consuming.

The core idea of BARRED is to eliminate reliance on large-scale manual annotations through automated synthetic data generation. Its dual guarantee mechanisms:

1. Dimension Decomposition

• Identify key dimensions, combine and explore to generate diverse scenarios, focusing on boundary cases;

2. Multi-Agent Debate Validation

• Asymmetric debate (presenting arguments from different angles), iterative validation (multi-round convergence to consensus), quality filtering (retaining only high-confidence samples) to ensure label accuracy.

Experiments cover scenarios such as content moderation and compliance checks, with results showing:

• Small fine-tuned models consistently outperform proprietary large language models and specialized guardrail commercial models;
• Inference costs are far lower than large models, achieving both accuracy and efficiency improvements;
• Ablation studies confirm: Removing dimension decomposition reduces data diversity, while removing the debate mechanism increases label error rates—both are indispensable.

Synthetic Data Quality Control

• Semantic consistency check, diversity measurement, label confidence evaluation;

Debate Mechanism Design

• Agent role assignment (user/regulator/business perspectives), balance of debate rounds, consensus achievement mechanism;

Method Comparison

Method	Annotation Requirement	Accuracy	Inference Cost	Maintainability
General Safety Model	Low	Medium	Medium	High
Prompt Engineering	Very Low	Medium-Low	High	Low
Manual Annotation + Fine-Tuning	Very High	High	Low	Medium
BARRED Synthetic Data	Low	High	Low	High

Applicable Scenarios

• Rapid prototype development, domain migration, policy iteration, resource-constrained environments;

Deployment Best Practices

1. Carefully write policy descriptions;
2. Collect representative unlabeled samples;
3. Iteratively optimize dimension decomposition;
4. Establish manual validation processes for low-confidence samples;
5. Continuously monitor and update synthetic policies.

Current Limitations

• Synthetic data quality decreases under complex/subjective policies;
• Primarily optimized for English, multi-language performance to be verified;
• Insufficient coverage of extremely rare long-tail scenarios;

Future Directions

• Adaptive dimension learning;
• Human-machine collaborative annotation;
• Cross-modal expansion.

The BARRED framework combines dimension decomposition and multi-agent debate to solve the problem of scarce high-quality training data, enabling small models to outperform proprietary large models. For enterprises, it eliminates the barrier of large-scale annotation, allowing resource-constrained teams to build professional-grade guardrail systems, which will play an important role in AI safety and compliance.