Practical Comparison of Small Language Models: In-Depth Evaluation of Qwen 3, Llama 3.2, and Phi 3 on Resume Analysis Tasks

This evaluation conducts a multi-dimensional assessment of the performance of three mainstream small language models (Qwen3 1.7B, Llama3.2 1B, Phi3 3.8B) on resume analysis tasks. Its core purpose is to provide reference for model selection in edge deployment and cost-sensitive scenarios. The evaluation reveals that the relationship between model size and actual performance is non-linear: Phi3 leads in reasoning ability but has moderate speed; Llama3.2 is extremely lightweight but has limited capabilities; Qwen3 achieves a balance between speed and intelligence. Additionally, it finds a gap between benchmark test results and real-world experience—small models still need to collaborate with large models to handle complex tasks.

Test Task Selection

The resume analysis task requires completing sub-tasks such as identifying core strengths/weaknesses, ATS (Applicant Tracking System) friendliness assessment, pointing out missing skills, generating improvement suggestions, and providing recruitment recommendation opinions—simulating the actual decision-making process of HR to test the model's comprehensive capabilities.

Evaluation Dimension Setting

A total of 9 dimensions: response clarity, instruction compliance, reasoning quality, hallucination tendency, accuracy, practical value, response speed, ambiguity handling ability, and humanized understanding.

Qwen3 (1.7B): The Balanced Performer

Strengths: Fast response speed, excellent instruction compliance, structured output; Limitations: Generalization in in-depth technical analysis, easy repetition of opinions in long outputs.

Llama3.2 (1B): The Cost of Extreme Lightweight

Strengths: Extremely fast response speed, concise and non-redundant output; Limitations: Superficial analysis lacking depth, generic suggestions without targeting.

Phi3 (3.8B): The Reasoning King Among Small Models

Strengths: Strong reasoning ability (mining implicit information), specific and practical suggestions, low hallucination risk; Limitations: Moderate speed, occasional overconfidence.

Horizontal Comparison Table

Evaluation Dimension	Qwen3 (1.7B)	Llama3.2 (1B)	Phi3 (3.8B)
Response Speed	High	Extremely High	Moderate
Reasoning Ability	Moderate	Low	High
Instruction Compliance	Good	Average	Excellent
Detail Level	Moderate	Low	High
Hallucination Risk	Moderate	Moderate	Low
Practical Value	Good	Basic	Excellent

Scenario-Based Recommendations

• Mobile/edge devices: Choose Llama3.2 (for simple tasks);
• General productivity tools: Choose Qwen3 (balances performance and cost);
• Professional analysis assistant: Choose Phi3 (deploy on servers/high-performance hardware).

1. Non-linear Relationship Between Size and Performance: Phi3 (3.8B) performs far better than Llama3.2 (1B), while the gap between Qwen3 (1.7B) and Llama3.2 is small. Architecture optimization and training data quality are more important than parameter stacking;
2. Gap Between Benchmark Tests and Real-World Experience: Lab results cannot fully reflect performance in real scenarios; actual testing for specific scenarios is necessary;
3. Limitations of Small Models: Complex reasoning tasks still require collaboration between small and large models—large models handle complex tasks, while small models are responsible for high-frequency simple interactions.

Future Outlook

SLM development directions: Model compression technologies (quantization/pruning/distillation), efficient architectures (Mamba/RWKV), multi-modal/domain-specialized models.

Conclusion

Small language models are democratizing AI, making intelligent computing accessible to all. Model selection needs to align with scenario requirements, balancing capability, cost, and latency. We look forward to more "small but powerful" models emerging to drive AI to be everywhere.