Reasoning models don't just think longer—their internal trajectories are truly different

Recent research finds that when reasoning-trained language models solve difficult problems, the geometric characteristics of their internal hidden state trajectories have systematic differences from ordinary instruction-tuned models, and this difference is most pronounced in the code domain. This post will detail the background, methods, findings, and significance of this study.

In recent years, reasoning models represented by OpenAI's o-series and DeepSeek-R1 have demonstrated strong complex problem-solving abilities, often generating longer chains of thought. However, just from the length of generation, it's impossible to tell whether the model uses a different internal strategy or merely extends computational steps mechanically. The research team attempted to answer this core question by analyzing hidden state trajectories.

The research team designed an analytical framework to compare the performance of reasoning-trained models and instruction-tuned baseline models in three domains: competitive programming, mathematical reasoning, and Boolean satisfiability problems. The key innovation is the introduction of a "length correction" mechanism to separate geometric patterns related to problem difficulty; they tracked hidden state sequences, constructed high-dimensional trajectories, and analyzed attributes such as curvature and heterogeneity.

In the code domain, when reasoning-trained models face more difficult programming problems, the corrected trajectories are more "direct" (focused and efficient paths), and the local curvature heterogeneity is significantly reduced (more consistent and stable internal representation strategies). The baseline models do not have this optimization pattern, indicating that reasoning training changes the internal mechanism rather than just increasing computational load.

In mathematical reasoning and SAT problems, similar trends were observed, but the effect strength was weaker than in the code domain. The domain differences may be because programming tasks have more explicit structural features and richer intermediate verification points, while mathematical/logical problems involve more operations on abstract concepts, leading to more complex geometric structures of internal representations.

Behavioral annotation analysis shows that stronger corrected geometric coupling occurs simultaneously with strategy shifts and uncertainty monitoring. Linear probe tests in the prompt phase did not reproduce the separation phenomenon in the code domain, indicating that the special geometric characteristics of reasoning models are mainly manifested during the generation process.

This study establishes length correction as a prerequisite for generating trajectory analysis, provides empirical support for the existence of reasoning ability, and the significant effect in the code domain provides clues for targeted model optimization. In the future, we can explore the application of trajectory geometric analysis in model diagnosis, ability prediction, and training optimization to help build more reliable and interpretable AI systems.