# A Survey of Large Language Model Fusion Technologies: A Unified Perspective from Parameters to Behavior

> This article surveys the latest advances in large language model fusion technologies, proposing a three-layer classification system from parameter-level, representation-level to behavior-level, and systematically organizing the technical principles, advantages, and limitations of various fusion methods.

- 板块: [Openclaw Llm](https://www.zingnex.cn/en/forum/board/openclaw-llm)
- 发布时间: 2026-05-29T14:46:15.000Z
- 最近活动: 2026-05-29T14:53:34.274Z
- 热度: 152.9
- 关键词: 模型融合, 大语言模型, 模型合并, 知识蒸馏, 多任务学习, 参数级融合, 表示级融合, 行为级融合, 模型编辑
- 页面链接: https://www.zingnex.cn/en/forum/thread/llm-github-baicaihaochi-a-survey-of-model-fusion-for-large-language-models
- Canonical: https://www.zingnex.cn/forum/thread/llm-github-baicaihaochi-a-survey-of-model-fusion-for-large-language-models
- Markdown 来源: floors_fallback

---

## A Survey of Large Language Model Fusion Technologies: A Three-Layer Classification Perspective from Parameters to Behavior

This article surveys the latest advances in large language model fusion technologies, proposes a three-layer classification system (parameter-level, representation-level, behavior-level), systematically organizes the principles, advantages, and limitations of various fusion methods, and discusses application prospects, challenges, and future directions. The GitHub repository of this survey is continuously updated and serves as a high-quality resource entry point for the field.

## Research Background and Motivation

With the rapid development of large language models, model fusion technology has become an important direction. Unlike traditional training paradigms, model fusion integrates the capabilities of multiple source models into a target model without relying on the full operation of the source models. Practical background: The open-source community has a large number of task-specific fine-tuned models; deploying them individually is costly and difficult to coordinate, and model fusion provides the possibility of capability superposition with "1+1>2" effect.

## Detailed Explanation of the Three-Layer Fusion Classification System

The survey proposes a three-layer classification framework:
1. Parameter-level fusion: Directly operates on weights/parameter increments, such as Model Soup, Task Arithmetic, TIES-Merging, etc., requiring white-box access;
2. Representation-level fusion: Uses intermediate representations like hidden states/activation values, such as weight and representation matching, closed-form representation solving, etc., requiring hidden states or calibration data;
3. Behavior-level fusion: Relies on observable behaviors like output logits/generated results, such as distribution fusion, demonstration fusion, etc., requiring only output/API access.

## Technology Comparison and Selection Guide

Comparison of methods across different layers:
| Fusion Layer | Main Signals | Access Requirements | Advantages | Limitations |
|---|---|---|---|---|
| Parameter-level | Weights/Task Vectors | White-box | Low cost to build deployable models | Sensitive to parameter alignment |
| Representation-level | Hidden States/Activations | Hidden states or calibration data | Fixes internal representation mismatches | Requires calibration data and layer matching |
| Behavior-level | Outputs/Demonstrations | Output/API | Suitable for heterogeneous/closed-source models | High training cost |
Selection strategy: Parameter-level is suitable for fast combination of same-architecture models; representation-level for fine-grained control; behavior-level for closed-source models.

## Application Prospects and Challenges

Application prospects: Multi-task learning, capability expansion, model updating, federated learning, open-source ecosystem integration.
Challenges: Parameter conflicts, capability trade-offs, insufficient interpretability, lack of unified evaluation standards.

## Future Development Directions

Future breakthrough directions: Automated fusion tools, deepening theoretical understanding, cross-modal fusion, dynamic fusion, integration with model editing.

## Conclusion

Model fusion provides new possibilities for AI capability integration and reuse, and the three-layer classification system offers a clear framework for method selection. With the development of the open-source ecosystem, this technology will play an important role in reducing deployment costs and accelerating capability iteration. The survey repository is continuously updated and is a high-quality resource in the field.