"Cognitive Fatigue" Phenomenon in Large Language Models: University of South Carolina AI Institute Reveals Structural Degradation in Transformer Long Text Generation

The University of South Carolina AI Institute proposed the concept of "Cognitive Fatigue" to describe the performance degradation of autoregressive language models in long text generation, and developed a fatigue index that can be calculated in real-time during inference. The study also constructed the Chatsparent real-time monitoring and intervention system, providing a technical framework for improving long conversation experiences and AI system reliability.

During long conversations with large models like ChatGPT and Claude, response quality often declines (repetitive content, reduced instruction following, unstable output). This phenomenon is an inherent structural feature of autoregressive Transformer architectures when generating long sequences. The University of South Carolina AI Institute conducted a systematic study on this, formally defining it as "Cognitive Fatigue" and proposing a lightweight diagnostic tool for real-time monitoring during inference.

Cognitive Fatigue is defined as: measurable degradation in a model’s instruction-following ability, representation stability, and prediction calibration during a single inference session, caused by cumulative state drift (non-parametric change) from increasing sequence length during decoding. Core symptoms include:

1. Instruction following attenuation: deviation from original prompt constraints
2. Unstable representation: hidden state distribution drift, decreased semantic consistency
3. Abnormal entropy: output distribution entropy fluctuations, reflecting abnormal uncertainty changes

The Fatigue Index (FI) is a normalized, model-agnostic diagnostic metric calculated token-by-token during inference without retraining, integrating three signals:

1. Prompt attention attenuation: monitoring Transformer’s attention weight dispersion on the original prompt
2. Embedding drift: tracking systematic drift patterns of hidden layer representations
3. Entropy deviation: observing abnormal output distribution entropy fluctuations (excessive uncertainty or repetition)

Validation on nine models of different scales/architectures supports the universality of cognitive fatigue (an inherent feature of autoregressive generation). Key findings:

• Fatigue observed in all tested models, with varying degrees and forms
• Fatigue index highly correlated with human-assessed output quality decline
• Fatigue signals appear earlier than visible quality degradation, supporting early intervention
• Fatigue patterns differ across tasks (Q&A, summarization, creative writing)

The Chatsparent system (based on the fatigue index) was presented at AAAI 2026, with features:

1. Real-time visualization: displaying fatigue index change curves during conversations
2. Early warning: alerts before significant quality decline
3. Retraining-free intervention: dynamic adjustment of decoding parameters, prompt refreshing, context compression (no model weight modifications) This system implements a "detection-warning-intervention" closed loop to improve long conversation experiences.

The value of cognitive fatigue research:

• User level: rational prompt design (timely context reset, chunked long text processing)
• Developer level: new dimension for model evaluation (comparing long text generation stability)
• AI safety level: monitoring risks from reduced instruction following, building reliable systems
• Hardware optimization: terminating generation on performance decline to avoid resource waste

Current limitations:

1. Fatigue index requires access to internal model states, limiting applicability to closed-source API models (e.g., GPT-4)
2. More validation needed for task/domain-specific fatigue pattern differences
3. Intervention strategy effectiveness needs improvement (alleviate fatigue while maintaining coherence) Future directions:

• Develop black-box fatigue estimation methods
• Explore architectural improvements (e.g., dynamic attention mechanisms)
• Integrate fatigue monitoring into production-level LLM services