VERA: A Hierarchical Reasoning Architecture Based on Predictive Coding, Enabling AI to Have Auditable Thinking Processes

VERA proposes a new reasoning paradigm for language models. Through bidirectional hierarchical information flow and predictive error minimization, it achieves dynamic computation, real uncertainty quantification, and fully auditable reasoning processes, aiming to break through the fundamental limitations of traditional language models' unidirectional fixed-depth computation.

Current mainstream language models follow a unidirectional paradigm of input tokens → fixed-layer forward propagation → output tokens. They have fundamental flaws: inability to reflect on predictions, inability to iteratively coordinate beliefs at different abstraction levels, and inability to express true 'uncertainty'. VERA was born to break through this limitation.

VERA is based on the predictive coding theory in neuroscience and builds a hierarchical architecture:

• Hierarchical structure: Multiple abstraction layers are stacked, each layer maintains a generative model for the next layer
• Top-down prediction: Higher layers send prediction signals to lower layers
• Bottom-up error: Lower layers feed back prediction errors to higher layers
• Iterative convergence: Continuous iteration until the predictive error across all layers is minimized The bidirectional flow mechanism changes the traditional 'one-time' reasoning mode.

1. Dynamic computation: Computation terminates when layers converge. Simple problems are solved quickly, complex problems trigger automatic deep reasoning, and the stopping time is determined by internal consistency checks.
2. Real uncertainty quantification: Real uncertainty is quantified through cumulative predictive errors, which is crucial for high-risk scenarios (medical, legal).
3. Auditable reasoning process: Tracks the refinement of conclusions from high-level abstraction to low-level concreteness, as well as traces of bottom-up feedback correcting high-level assumptions, improving transparency.
4. Internal consistency check: Ensures logical self-consistency of outputs through layer coordination, solving the problem of contradictions in traditional models.

VERA uses the GPL-3.0 open-source license and is in an active development phase. Milestones include:

• Phase 2→3: Convergence rate of in-distribution samples within 32 iterations ≥80%
• Phase 3→4: Correlation coefficient between problem difficulty and number of iterations >0.5
• Phase 4→5: Accuracy of identifying the source layer of problems via hierarchical free energy decomposition >60%
• Phase5→Evaluation: Success rate of adversarial attacks controlled below 5% Phased verification ensures the gradual establishment of the architecture's reliability.

• Content safety: The v0.0 version has no instruction-level filtering; content policies are the responsibility of the deployer.
• Alignment issues: The current version does not implement alignment methods such as RLHF or Constitutional AI; these will be considered in v1.x versions.
• Architectural safety: Safety attributes are geometric and structural, relying on the mathematical principles of predictive coding to hold. The team openly notes current limitations and future directions.

VERA represents a paradigm shift: Transformers enabled AI to learn 'pattern matching', while VERA attempts to make AI learn 'thinking' (an iterative process of establishing consistency between multi-level representations). If successful, it may bring:

1. More reliable AI (able to express 'I don't know')
2. More efficient reasoning (resource allocation on demand)
3. More interpretable AI (transparent and traceable reasoning process)
4. Safer deployment (internal consistency checks catch errors) VERA is in its early stages but has great potential. It returns to the first principles of cognitive science to rethink the nature of reasoning, and is worthy of attention in the fields of AI architecture, interpretability, and safety.