RD-Net: Resolving Repetition Collapse in Long Text Generation of Large Language Models via a Drift Mechanism

RD-Net is a solution to the repetition collapse problem in long text generation of Large Language Models (LLMs) using an innovative drift mechanism. It does not require fine-tuning frozen models, is plug-and-play, can significantly reduce repetition, maintain the coherence of generated content, and provide effective support for the application of LLMs in long text scenarios.

LLMs perform well in short text generation, but long text generation is prone to repetition collapse (repeating phrases, sentences, or paragraphs), which affects output quality and limits their application in scenarios such as novel writing and academic paper composition. Traditional solutions like fine-tuning or post-processing are either high-cost or have limited effectiveness.

The core of RD-Net is the drift mechanism: introducing controlled drift during the generation process, slightly adjusting the model's internal state, guiding the model to explore new expression spaces, avoiding repetitive loops, while maintaining content coherence and semantic consistency without the need for model fine-tuning.

RD-Net is applicable to frozen LLMs and has the advantages of no fine-tuning required, plug-and-play, model-agnostic, and low overhead. The core code is encapsulated in Python's rd_wrapper.py, which can seamlessly wrap existing inference processes without changing the original code structure.

RD-Net performs significantly in long text generation tasks: it reduces the proportion of repeated n-grams, maintains semantic coherence and logical consistency; as the generation length increases, it still maintains stable generation quality with low repetition rates in tasks of thousands of tokens.

RD-Net is suitable for scenarios such as creative writing (long stories), academic writing (literature reviews), technical documentation (API manuals), dialogue systems (long conversation responses), and code generation (long code snippets).

Traditional methods such as temperature adjustment (sacrificing coherence), Top-p/Top-k sampling (limited effect on long texts), and repetition penalty (semantic deviation) have shortcomings. RD-Net balances diversity and coherence through an intelligent adaptive drift mechanism, achieving better results.

RD-Net provides a simple and effective solution to the repetition collapse problem in LLM long text generation, improving generation quality and expanding application scenarios. Future directions will include exploring adaptive drift, multilingual support, integration with Retrieval-Augmented Generation (RAG), and theoretical analysis.