RLVR Reasoning Training Data Allocation Strategy: A Study on Dual-Dimensional Control of Reasoning Depth and Environmental Complexity

This study focuses on data allocation strategies for RLVR reasoning training. By constructing a synthetic knowledge graph environment, it systematically analyzes the impact of two dimensions—reasoning depth and environmental complexity. Key findings include: data allocation strategies covering both dimensions jointly outperform single-axis schemes; inductive-analogical and deductive-abductive reasoning form two distinct task clusters; strategies that uniformly mix samples of different difficulty levels perform better. This research provides key design principles for enhancing the comprehensive reasoning capabilities of models.

RLVR (Reinforcement Learning with Verifiable Rewards) has become a mainstream post-training method for enhancing the reasoning capabilities of large language models, significantly improving performance on tasks like mathematics and coding. However, existing studies have limitations: they have a single-dimensional understanding of the reasoning space, equating difficulty only with reasoning depth, while ignoring the multi-dimensional complexity of real-world reasoning (e.g., environmental interference, multi-path filtering, etc.).

Characterization of Dual-Dimensional Reasoning Space

1. Difficulty Dimension: Expanded to reasoning depth (length of reasoning chain) + environmental complexity (distractors and path filtering)
2. Reasoning Forms: Covers four core capabilities: deduction (forward reasoning), abduction (reverse explanation), induction (pattern discovery), and analogy (knowledge transfer)

Synthetic Knowledge Graph Environment

Construct a controllable environment to precisely control parameters such as pre-training/post-training data distribution, reasoning depth, and environmental complexity, eliminating confounding factors in real data and supporting controlled experiments.

Finding 1: Joint Dimension Coverage is Superior

Strategies covering both reasoning depth and environmental complexity simultaneously significantly outperform single-dimensional schemes (avoiding imbalance between mechanical reasoning and information extraction capabilities).

Finding 2: Reasoning Task Clustering

The four reasoning forms form two clusters: deductive-abductive reasoning as one cluster, inductive-analogical as the other; abductive reasoning is more sensitive to training coverage—performance drops sharply when coverage is insufficient.

Finding 3: Uniform Mixing Strategy is Better

With a fixed budget, strategies that uniformly sample samples of different difficulty levels outperform phased curriculum learning (providing richer signals and avoiding adaptation costs).

Testing open-source/closed-source models reveals that existing models generally exhibit an asymmetry where deductive reasoning outperforms abductive reasoning. This reflects a systemic bias in training data—overrepresentation of deductive tasks and underrepresentation of abductive tasks—limiting the models' applications in fields like scientific discovery and fault diagnosis.

1. Multi-Dimensional Data Evaluation: Use a multi-dimensional framework (reasoning depth + environmental complexity) to evaluate data difficulty
2. Balanced Reasoning Coverage: Deliberately balance training data across the four reasoning forms (deduction, abduction, induction, analogy)
3. Redesign Curriculum: Consider uniform mixing strategies instead of traditional phased curricula
4. Focus on Abduction: Design specialized enhancement strategies or evaluation benchmarks targeting the vulnerability of abductive reasoning

Limitations

• The correspondence between synthetic environments and real tasks needs verification
• Experiments are limited to small and medium-sized models; need to extend to large models
• Insufficient exploration of extremely long reasoning chains (>100 steps)

Future Directions

• Verify findings on real datasets
• Explore more dimensions for characterizing the reasoning space
• Develop adaptive data allocation algorithms

Through controlled experiments, this study expands the reasoning space from one dimension to two, revealing key principles for RLVR data allocation. Its core contribution is proving the necessity of multi-dimensional data curation (joint depth and complexity, balanced reasoning types) for cultivating comprehensive reasoning capabilities, providing direct guidance for reasoning training of AI systems.