Deployment Practice of Three Domestic Open-Source Large Models: Performance Comparison and Semantic Understanding Evaluation of Qwen, ChatGLM3, and Baichuan2

This article deploys three domestic open-source large models—Qwen-7B-Chat, ChatGLM3-6B, and Baichuan2-7B-Chat—on the ModelScope GPU Notebook platform. Through 5 challenging Chinese semantic test questions, it conducts a horizontal comparison across five dimensions: memory usage, model structure, algorithm highlights, actual performance, and applicable scenarios, providing a reference for developers to select the appropriate large model. Original author: Evan-Lii; Source: GitHub; Publication date: May 31, 2026.

With the development of the open-source large language model ecosystem, domestic models have unique advantages in Chinese semantic understanding. As an assignment for an AI introduction course, this project selects three 7B/6B-level open-source dialogue models (Qwen-7B-Chat, ChatGLM3-6B, Baichuan2-7B-Chat) for deployment testing and capability evaluation. The core objectives are to complete local deployment and evaluate the models' differences in capabilities such as ambiguity understanding and nested logical reasoning through 5 Chinese semantic test questions, forming a selection guide covering hardware adaptation to application scenarios.

Hardware configuration: CPU Notebook (cloud virtualization), 8 vCPUs, 32GB RAM, cloud SSD storage + high-speed network. Software environment: Ubuntu 22.04 image, Python 3.10, torch 2.3.0+cpu, transformers 4.33.3, modelscope 1.9.5. The configuration is optimized for inference of 7B/6B models, enabling smooth operation through quantization and memory management strategies.

• Qwen-7B-Chat (Alibaba DAMO Academy): Transformer decoder architecture, 32K context window, deeply optimized for Chinese, top-ranked in multiple Chinese NLP benchmarks, excels at complex semantic reasoning.
• ChatGLM3-6B (Zhipu AI): GLM architecture (autoregressive fill-in-the-blank pre-training), 6B parameters with performance close to 7B models, supports tool calling and multimodal understanding, and has comprehensive functions.
• Baichuan2-7B-Chat (Baichuan Intelligence): Transformer architecture, strictly screened training data to enhance Chinese understanding and safety alignment, with built-in multi-layer safety filtering mechanisms.

Five types of Chinese semantic challenges are designed: 1. Seasonal clothing ambiguity (opposite meanings of "wear as much as possible" in winter vs. summer); 2. Pun semantics (e.g., double interpretation of "nobody looks up to" or "can't look up to anyone"); 3. Multi-layer nested logic (e.g., "Do you know the thing that I don't know you know?"); 4. Name semantic ambiguity (distinguishing proper nouns from common words); 5. Implicit intent inference (capturing implied meanings).

Comparison dimensions: 1. Memory usage and resource efficiency (evaluating deployability in resource-constrained environments); 2. Model architecture and parameter efficiency (analyzing the impact of architecture design on performance); 3. Algorithm innovation and optimization highlights (innovations in pre-training/fine-tuning/alignment technologies); 4. Actual inference performance (accuracy in semantic understanding/logical reasoning, etc.); 5. Applicable scenarios and selection recommendations (recommendations for customer service/content creation, etc.).

7B/6B-level models are currently practical deployment choices, and each of the three models has its own advantages: Qwen excels in Chinese understanding and reasoning, ChatGLM3 has comprehensive functions, and Baichuan2 stands out in safety alignment. Selection should consider scenarios, hardware, and safety requirements. With future advancements in model compression, inference acceleration technologies, and accumulation of Chinese data, domestic models are expected to make breakthroughs in more vertical fields and promote the popularization of AI.