CoT-Loop: Detecting Cyclic Behavior in Large Model Reasoning

CoT-Loop is an open-source project that studies cyclic generation behavior in the chain-of-thought (CoT) reasoning of large language models (LLMs). By analyzing the model's internal activation states and reasoning trajectories, it attempts to predict and detect the risk of the model falling into infinite loops, with the goal of improving the reliability and safety of AI systems.

As LLMs' capabilities in complex reasoning tasks improve, CoT prompting has become a key technology. However, models tend to fall into infinite loops (repeatedly generating similar reasoning steps without convergence), similar to humans "getting stuck in a rut". This not only affects user experience but also wastes computing resources and causes response delays. The core question of the CoT-Loop project: Can we predict the risk of loops from the model's internal activations and generation trajectories?

CoT-Loop adopts two complementary research lines:

1. Pre-filling Probe: Extract the stacked activations of the last token in the final layer during the prompt pre-filling phase, train a binary classifier to predict loop risks, compare single-layer activation and cross-layer voting strategies, and the full-layer last-token anchor is the optimal pre-filling scheme;
2. Reasoning Statistics: Collect generation statistics across multiple benchmark tests (MATH-500, AIME, etc.), and use a unified decoding strategy (temperature=0.2, generate 10 samples per prompt) to ensure comparable results.

Loop Definition: A loop is marked if any 30-gram in the generated sequence appears 20 times or more (parameters can be adjusted). Implementation Process:

1. Construct formatted chat prompts for the model;
2. Extract the stacked last-token activations from the pre-filled state;
3. Generate reasoning trajectories and label loop/non-loop samples;
4. Train a binary probe classifier;
5. Evaluate the prediction accuracy of the probe.

Technical Details:

• Dataset Construction: Extract feature labels via scripts/build_probe_dataset.py, support multiple model presets, and store the last_token_all_layers_stack_final feature by default;
• Probe Training: scripts/train_probe.py supports linear and mlp probes (mlp by default), records training metrics, and saves the best checkpoint;
• Model Presets: Predefined model configurations such as qwq_32b and openthinker3_7b, including TP/DP/temperature/max token count, which can be manually overridden.

Key Findings:

1. Limitations of Pre-filling Probe: The full-layer last-token anchor is the optimal pre-filling scheme, but methods based on a complete generation view are better;
2. Adjustment of p_loop Objective: It is no longer the default training objective, and related definitions have been moved to the documentation;
3. Metadata Control: The original association table has been replaced by a training metadata control package. Current Work Focus: Shift to executive tasks such as full training under fixed architecture and restoring necessary accuracy tables.

Significance for AI Safety and Reliability:

1. Runtime Risk Warning: Predict loop risks in advance, allowing adjustment of decoding parameters, addition of anti-loop prompts, or model switching;
2. Model Evaluation and Selection: Loop occurrence rate can be used as a model quality indicator to assist selection decisions;
3. Prompt Engineering Optimization: Understand the correlation between prompt features and loop risks to design more robust prompt templates.

CoT-Loop represents an important direction in AI interpretability research. By combining internal activation analysis and external behavior statistics, it provides a new perspective for understanding the reasoning mechanism of LLMs. Although loop detection still faces challenges, the project demonstrates a feasible path and is expected to build more reliable and predictable AI reasoning systems, making CoT a tool for solving problems rather than a trap.