DistillTableCoT: A Chain-of-Thought Knowledge Distillation Framework for Table Reasoning

DistillTableCoT is an experimental project that explores distilling the chain-of-thought (CoT) reasoning capabilities of large language models (LLMs) into small models (Qwen3-1.7B), aiming to provide a lightweight solution for table question answering (TableQA) scenarios. The project supports multiple table tasks (such as WikiTableQA, TabFact, etc.) and hybrid reasoning paradigms (combining SQL-Agent and CoT), reducing computational overhead while maintaining reasoning quality.

Table data is widely used in finance, science, business, and other fields, but enabling AI to understand tables and perform reasoning-based question answering (TableQA) is a difficult problem in NLP. While large language models have strong table understanding capabilities, their large parameter size and high reasoning cost limit practical deployment. To address this issue, DistillTableCoT explores distilling the CoT capabilities of large models into the Qwen3-1.7B small model, balancing reasoning quality and computational efficiency.

The project supports multiple table reasoning tasks:

• WikiTableQA: Answer natural language questions based on tables (e.g., comparing national populations);
• TabFact: Determine whether a statement is consistent with table content (fine-grained fact-checking);
• FeTaQA: Generate free-form answers (non-extractive, closer to real-world scenarios);
• HiTab: Process hierarchical tables (multi-level headers) and perform numerical calculations (testing complex structure parsing and multi-step reasoning).

Hybrid Reasoning Paradigm: Combining end-to-end generation and SQL queries, supporting three modes:

• SQL-Agent mode: Generate SQL queries to obtain results and then generate answers;
• CoT mode: Multi-step reasoning for gradual deduction;
• Hybrid mode: Dynamically select or combine the two paradigms.

Distillation Process:

1. Teacher model reasoning: Use large models such as DeepSeek API to generate CoT-included reasoning processes and answers;
2. Data preparation: Process teacher outputs into a standard format (question, table, reasoning, answer);
3. Student model training: Fine-tune Qwen3-1.7B (supports pure CoT, SQL-Agent, or hybrid mode);
4. Evaluation and validation: Evaluate the effects of different distillation strategies using test sets.

Code Structure: Modular design with key directories including configs (configuration), data_loader (data loading), scripts (training scripts), etc.

Usage Steps:

1. Environment preparation: Python3.8+, install dependencies (pip install -r requirements.txt), download NLTK data packages;
2. Generate teacher data: Run run_inference.py to call large models and generate results;
3. Train the student model: Run train_distill.py using a YAML configuration file;
4. Test and evaluation: Use test_model.py to evaluate the trained model.

• Lightweight Deployment: Distill capabilities into a 1.7B small model, which can be deployed on edge devices, mobile terminals, or cost-sensitive scenarios;
• Domain Adaptation: Quickly adapt to specific vertical domains by re-distilling with domain data;
• Hybrid Reasoning Inspiration: The paradigm combining SQL-Agent and CoT can be extended to knowledge graph question answering, database natural language interfaces, and other scenarios.

Limitations:

• Limited dataset coverage; needs to be extended to more actual table forms;
• Insufficient capabilities for complex reasoning (multi-table join, complex calculations);
• Generalization ability depends on the quality of the teacher model and distillation data.

Future Directions:

• Introduce diverse table data sources;
• Explore advanced technologies such as contrastive learning and adversarial distillation;
• Support multi-modal table understanding (combining visual layout);
• Optimize reasoning efficiency to support real-time interactive question answering.

DistillTableCoT provides a complete experimental framework for knowledge distillation in table reasoning tasks, demonstrating methods for transferring CoT capabilities from large models to small models and exploring hybrid reasoning paradigms. For developers and researchers who want to deploy TableQA functions in resource-constrained environments, this project has important reference value.