# Systematic Evaluation of Nine Prompt Strategies in Commonsense Reasoning Tasks

> This article deeply analyzes an open-source project that comprehensively evaluates nine different prompt strategies (including zero-shot chain-of-thought, few-shot chain-of-thought, Plan-and-Solve, Tree-of-Thought, etc.) on the CommonsenseQA dataset, using the DeepSeek-R1-Distill-Qwen-7B model for testing.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-04-30T09:11:40.000Z
- 最近活动: 2026-04-30T09:17:34.832Z
- 热度: 150.9
- 关键词: 提示工程, 思维链, 常识推理, LLM评估, DeepSeek, CommonsenseQA, Tree-of-Thought, Self-Consistency
- 页面链接: https://www.zingnex.cn/en/forum/thread/llm-github-zak-hanfer-commonsense-qa-with-chain-of-thought
- Canonical: https://www.zingnex.cn/forum/thread/llm-github-zak-hanfer-commonsense-qa-with-chain-of-thought
- Markdown 来源: floors_fallback

---

## [Introduction] Core Overview of Systematic Evaluation of Nine Prompt Strategies in Commonsense Reasoning Tasks

This article introduces an open-source project that conducts a comprehensive comparative test of nine mainstream prompt strategies (including zero-shot chain-of-thought, few-shot chain-of-thought, Plan-and-Solve, Tree-of-Thought, etc.) on the CommonsenseQA dataset, using the DeepSeek-R1-Distill-Qwen-7B model (7B parameters, inference-optimized). The core goal is to explore which prompt strategy achieves the best performance in commonsense reasoning tasks under resource-constrained environments, providing empirical data for prompt engineering practices.

## Project Background and Objectives

Prompt engineering has a significant impact on LLM performance, especially in tasks like commonsense reasoning that require going beyond pattern matching. CommonsenseQA is a classic commonsense reasoning benchmark built on ConceptNet, containing 12,154 training samples and 1,221 development samples, with 5 options per question. The core question of the project: Which prompt strategy works best under resource constraints (7B model)? The DeepSeek-R1-Distill-Qwen-7B model was chosen for testing because it balances reasoning ability, speed, and hardware accessibility.

## Detailed Explanation of Nine Prompt Strategies

1. Baseline method: Directly ask the question without reasoning guidance, used as a control;
2. Zero-shot chain-of-thought: Add "Let's think step by step" to guide the generation of intermediate steps, no examples needed;
3. Few-shot chain-of-thought: Provide examples with complete reasoning for in-context learning;
4. Restate and Expand (RE): Restate the question first before answering to reduce understanding bias;
5. RE+: Add additional context on top of RE;
6. Plan-and-Solve (PS): Split into planning and execution phases, suitable for complex problems;
7. PS+: Add explicit constraints on top of PS;
8. Tree-of-Thought (ToT): Explore multiple reasoning paths;
9. Self-consistency: Aggregate results via voting after multiple independent reasoning attempts to improve reliability.

## Evaluation Methods and Experimental Design

Experimental process:
1. Load the CommonsenseQA dataset using the Hugging Face datasets library;
2. Design prompt templates for each strategy;
3. The model receives formatted questions and options, generating answers with reasoning;
4. Parse the output to extract answers, compare with standard answers to calculate accuracy;
5. Save results as JSON and summarize into CSV reports for easy analysis and visualization.

## Key Findings and Practical Insights

Main conclusions:
1. Structured prompts (such as chain-of-thought, PS, ToT) are significantly better than the baseline;
2. In few-shot CoT, the quality of examples is more important than quantity;
3. Self-consistency has high cost-effectiveness, and voting aggregation improves reliability;
4. The PS strategy performs outstandingly in multi-step reasoning.

## Application Scenarios and Limitations

Application scenarios: Edge-side LLM deployment (maximizing performance with limited computing power), prompt engineering research (providing comparative data), educational AI (designing effective tutoring systems).
Limitations: Only tested English commonsense, results may not apply to other languages/domains; based on a 7B model, conclusions may not be applicable to larger-scale models.

## Conclusion and Future Directions

The value of the project lies in providing a reproducible evaluation framework to facilitate the objective assessment of new prompt strategies. Key insight: There is no one-size-fits-all strategy; selection must be based on specific tasks, model scale, and performance requirements. Prompt engineering requires making informed choices based on method characteristics rather than looking for a silver bullet.