DPC: A Training-Free Dual-Paradigm Consistency Framework for Text-to-SQL Candidate Selection

DPC (Dual-Paradigm Consistency) is a training-free Text-to-SQL candidate selection framework that corely addresses the "generation-selection gap" problem of LLMs in Text-to-SQL tasks. It achieves an accuracy improvement of up to 2.2% by constructing a Minimal Distinction Database (MDD) and a cross-paradigm validation strategy. This thread will introduce its background, methods, experimental results, and application prospects in separate floors.

The Generation-Selection Gap in Text-to-SQL

Although LLMs can generate SQL, they lack self-evaluation ability without an execution environment, leading to high Pass@K but low Pass@1 (only one can be selected in practical applications), forming the "generation-selection gap."

Dilemmas of Existing Solutions

• Supervised Verifiers: Require large amounts of labeled data, prone to overfitting, and have poor generalization;
• Self-Consistency: If the model has systematic biases, it is prone to "consensus hallucination";
• LLM-as-a-Judge: Lack symbolic execution ability and are easily misled by surface features.

The core idea of DPC is to transform SQL selection from probabilistic guessing into deterministic verification with visible data. It adopts a three-agent collaboration architecture:

• SLICER: Analyzes the structure of SQL candidates and identifies key differences;
• TESTER: Constructs a Minimal Distinction Database (MDD) based on the differences;
• SOLVER: Performs cross-paradigm validation (converts SQL to Python/Pandas) and compares the consistency of execution results.

Minimal Distinction Database (MDD)

MDD is a key innovation of DPC, with the following features:

• Adversarial design: Constructed in a targeted manner to maximize candidate differences;
• Fully observable: Data is visible and controllable, with transparent verification;
• Minimal scale: Contains only the minimum data needed for distinction, reducing costs;
• Logic-focused: Precisely exposes differences in boundary conditions, null value handling, etc.

Cross-Paradigm Validation

Implemented by converting SQL to Python/Pandas:

• Paradigm independence: Declarative (SQL) and imperative (Python) syntax have different thinking patterns; if results are consistent, the credibility is high;
• Bias offset: Different paradigms are less likely to have the same systematic biases;
• Executable verification: Pandas provides a reliable execution foundation.

Experimental Results

Evaluated on mainstream LLMs using BIRD and Spider benchmarks:

• Accuracy improvement: 2.2% higher than baselines like Self-Consistency;
• Consistency advantage: Stable across different models and query difficulties;
• Training-agnostic: No training required, with strong generality.

Ablation Experiments

• Necessity of MDD: Replacing MDD with random data leads to a significant performance drop;
• Value of cross-paradigm: Single-paradigm verification has low accuracy;
• Multi-agent collaboration: Individual agents cannot achieve the performance of the complete system.

Technical Insights

• From guessing to verification: Constructing a controllable test environment is more reliable than relying on probabilistic outputs;
• Cross-paradigm redundancy: Cross-validation reduces the risk of systematic biases;
• Agent specialization: Decomposing tasks to specialized agents improves performance;
• Interpretability: The verification process is transparent, and decision reasons can be traced.

Application Prospects

• Code generation: Can be extended to other executable code scenarios;
• Formal verification: Explore high-reliability fields by combining with symbolic execution;
• Multi-agent systems: The collaboration architecture can be extended to more AI scenarios.

Limitations

• MDD construction has computational overhead; complex queries may lead to state space explosion;
• Cross-paradigm validation requires a reliable execution environment for the target language.

Conclusion

DPC provides a new idea for solving the Text-to-SQL generation-selection gap, and its method combining MDD and cross-paradigm validation is of great significance in building reliable intelligent systems.