RD-Net: Solving the Repetition Collapse Problem in Long Text Generation with Large Language Models

This article introduces the RD-Net project, which effectively reduces repetition collapse in long text generation by frozen large language models (LLMs) through a simple drift mechanism. It does not require model fine-tuning or retraining, providing a practical solution to improve LLM generation quality.

Large language models (LLMs) perform excellently in short text generation tasks, but often encounter repetition collapse when generating long content—constantly repeating the same phrases, sentences, or concepts, leading to decreased output quality and poor readability. The root causes lie in the attention mechanism and autoregressive generation method: as text length increases, the model gradually "loses itself" in its own output, unable to maintain content diversity and coherence, which poses a serious obstacle to scenarios such as long reports, novels, and technical documents.

The core of RD-Net (Representational Drift Network) is the drift mechanism, which can be directly applied to frozen pre-trained models. Its working principles include: 1. Monitoring representation changes: tracking the trajectory of hidden layer state changes in the model; 2. Detecting repetition signals: identifying convergence trends in the representation space (a precursor to repetition collapse); 3. Dynamic adjustment: breaking convergence through slight perturbations to guide the model to explore new generation directions. This method is similar to adding "creative noise" to prevent the model from falling into locally optimal repetition loops.

RD-Net focuses on practicality and ease of use, supporting cross-platform deployment:

System Requirements

• Operating System: Windows 10+, macOS 10.12+, or mainstream Linux distributions
• Memory: At least 4GB RAM
• Storage: Minimum 500MB of available space

Core Function Modules

• Drift Detection Engine: Real-time analysis of representation vectors and calculation of drift metrics
• Feedback Adjustment System: Dynamically adjust generation strategies based on the degree of drift
• Performance Log Module: Record generation metrics for analysis and optimization
• Intuitive User Interface: Simplify configuration and monitoring to lower the threshold for use

RD-Net technology can be widely applied in:

1. Long Document Generation: Maintain diversity and coherence in content such as technical documents and research reports;
2. Creative Writing Assistance: Help AI maintain the richness of narratives and plot progression in novels and scripts;
3. Dialogue System Optimization: Prevent chatbots from repeating response patterns and improve user experience;
4. Code Generation: Avoid repeated similar code blocks and enhance the practicality and diversity of generated code.

Although there is no detailed quantitative evaluation data, from the technical principle perspective, RD-Net has the following theoretical advantages:

1. No Training Cost: Directly applied to frozen models, saving computing resources;
2. Strong Generality: Adaptable to different LLM architectures (e.g., Llama series);
3. Real-time Response: Intervene instantly during generation without affecting user experience;
4. Interpretability: Drift metrics provide visual insights into model behavior.

Usage and Deployment

Users can download the corresponding platform installation package from GitHub Releases, install it according to the instructions, and run it via command line or graphical interface; the open-source code supports deep customization, allowing adjustment of drift detection sensitivity and intervention strategies.

Technical Prospects

RD-Net represents an important direction for LLM optimization: improving generation quality through external mechanisms without changing the model itself. Its strategic significance includes: reducing deployment costs (no need to retrain the base model), enabling rapid iteration (algorithm optimization is independent of model updates), and composability (can be stacked with technologies like speculative decoding and quantization). As LLMs expand to long text scenarios, such stability-enhancing technologies will become key components driving the practical application of large models.