Decide First, Think Later: The Phenomenon of "Post-hoc Rationalization" in the Chain of Thought of Reasoning Models

Studies indicate that reasoning models encode decision outcomes before generating the chain of thought; the thinking process is often a rationalization of pre-determined decisions rather than genuine reasoning. Using linear probing and activation intervention techniques, the internal decision-making mechanisms of models have been revealed, challenging traditional perceptions of the chain of thought.

Descartes' "I think, therefore I am" extended to AI: Do models think first then decide, or vice versa? Traditionally, the chain of thought is seen as a transparent reasoning process and a basis for trusting AI, but recent research has raised doubts.

Using linear probing technology (training a classifier on the model's internal activation states to predict outputs), it was found that tool-use decisions can be predicted before generating any reasoning tokens—even before the first reasoning token, the model has already "made up its mind". The high accuracy indicates that decisions are encoded early.

Through activation intervention (modifying internal activations to observe behavior), after perturbing in the direction of a decision, when the model's decision changes, the chain of thought changes accordingly—but this is not re-reasoning, but rather finding justifications for the new decision. Interventions can reverse 7%-79% of the model's behavior, and the chain of thought after the change often involves "overthinking".

The chain of thought is more like a narrative generator that weaves reasonable explanations for already-made decisions, similar to human confirmation bias and post-hoc rationalization—after the model internally encodes a decision, reasoning tends to reinforce rather than question that decision.

This poses challenges to credibility in high-risk fields (such as healthcare and law), as the chain of thought may mask biases/errors. Technically, this stems from the separation of decision-making and explanation subsystems in the Transformer architecture, along with a lack of effective feedback mechanisms.

1. Design new training objectives (e.g., "diversity of thought" rewards) to encourage open reasoning; 2. Develop detection mechanisms to identify signs of post-hoc rationalization; 3. Explore new architectures where decision-making and explanation are closely coupled.

AI exhibits cognitive biases similar to humans, sparking reflections on the decision-making mechanisms of intelligent systems. In the future, we need to ensure that the chain of thought truly reflects the cognitive process rather than being a post-hoc packaging, to build trustworthy AI.