IS-CoT: Breaking Performance Collapse in Long-form Text Generation via Interleaved Structural Thinking

Large language models face performance collapse when generating long-form text. The IS-CoT framework embeds a dynamic plan-write-reflect cycle to achieve continuous strategy adaptation and global alignment without external assistance, outperforming DeepSeek-V3.2 by 3.08 points in benchmarks like LongBench-Write. Original paper source: arXiv, title IS-CoT: Breaking the Long-form Generation Collapse via Interleaved Structural Thinking, link http://arxiv.org/abs/2606.09709v1, published on 2026-06-08.

Large language models (LLMs) perform well in logic-intensive tasks, but there is a 'length collapse' phenomenon in open-ended long-form writing—when the target text exceeds 2000 words, performance drops sharply, and content lacks coherence and controllability. The root cause lies in the insufficiency of static hierarchical planning mechanisms: once an outline is made at the initial stage of generation, it is not adjusted anymore, making it impossible to dynamically correct and difficult to meet the needs of long texts with multi-paragraph connections.

The IS-CoT (Interleaved Structural Thinking) framework embeds a dynamic plan-write-reflect cycle into the generation process, which is an endogenous mechanism without external tools. The core innovation is 'interleaving': traditional methods execute planning, writing, and reflection in phases, while IS-CoT allows the three to alternate at the micro level—after generating each paragraph, it evaluates the fit with the overall goal and fine-tunes the subsequent plan to ensure the global consistency of long texts.

To train the IS-Writer-8B model, the team built a high-quality dataset containing a large number of interleaved reasoning trajectories, and used a multi-teacher pipeline to integrate the advantages of multiple advanced models to screen samples. The training focus is not only on 'what to write' but also on learning 'how to plan writing' and 'when to adjust strategies', cultivating metacognitive abilities to adapt to different length requirements.

In benchmarks like LongBench-Write, the IS-Writer-8B (8 billion parameters) performs leadingly, improving by 3.08 points compared to DeepSeek-V3.2, and can compete with larger proprietary models. In addition, the model can accurately follow user-specified length requirements, neither ending prematurely nor over-generating, demonstrating excellent length compliance.

The success of IS-CoT shows that the key to improving the quality of long-form text generation is not expanding the model scale, but optimizing the dynamic decision-making mechanism in the generation process. Embedding reflection capabilities into the generation process provides a new direction for model architecture design. For developers and researchers, IS-CoT provides a reference paradigm: through training on structured thinking trajectories, smaller models can also break through long-form text tasks.