Dual-Perspective Analysis of Self-Referential Representations in Large Language Models: Integration of Biological Topology and Activation Space Geometry

This article proposes an innovative interpretability research method that combines biological topology and activation space geometry to characterize self-referential representations in large language models from dual dimensions, providing a new perspective for understanding the internal mechanisms of models. Keywords: Interpretability, Large Language Models, Self-Reference, Biological Topology, Activation Space Geometry, etc.

Interpretability of large language models is a cutting-edge topic in the AI field. The expansion of model scale leads to an exponential increase in the complexity of internal representations, making it difficult for traditional single methods to capture deep mechanisms. Self-reference is a core feature of intelligent systems, supporting advanced functions such as metacognition. Understanding its formation and application in large models is key to revealing the essence of intelligence. Existing methods mostly focus on a single dimension (neural topology or geometry), limiting the comprehensive grasp of complex structures.

Biological Topology Perspective: Drawing on topological analysis methods in neuroscience, it focuses on connection topology (attention head connection patterns), hierarchical topology (inter-layer information flow), and functional topology (neurons/attention heads involved in specific computations). It identifies stable functional modules through topological invariants (Betti numbers, persistent homology).

Activation Space Geometry Perspective: Focuses on the structure of the representation vector space, including representation manifolds, decision boundaries, vector arithmetic, etc. It particularly pays attention to the encoding of self-referential concepts in the activation space and their geometric relationships with other concepts.

Data Preparation and Experimental Design: Construct self-referential corpora (different styles/contexts), record activation values of key layers, and compare activation patterns of non-self-referential texts.

Topological Analysis Process: Build similarity networks → Calculate topological features (persistent homology) → Identify functional modules (community discovery) → Cross-layer comparison.

Geometric Analysis Process: Dimensionality reduction visualization (t-SNE/UMAP) → Calculate geometric metrics (distance, angle) → Subspace analysis → Causal intervention verification.

Dual Characteristics of Self-Referential Representations: From the topological perspective, it presents a "core-edge" structure (a small number of core neurons + peripheral context regulation), similar to the topology of the human default mode network. From the geometric perspective, it forms a compact cluster and maintains a specific angular relationship with other concepts, endowing it with a special representational status.

Implications: Representations are hierarchical (low/middle/high layers handle lexical grammar, semantic context, and self-model integration respectively); computations are distributed (collaboration of multiple components); there is a balance between emergence and construction (abilities emerge but are rooted in training data and architecture).

Potential Applications: Model safety assessment (identifying risks of harmful self-reference), capability assessment benchmarks (supplementing dimensions of intelligence level), model editing and alignment (adjusting self-cognition).

Limitations: The causal relationship between topological/geometric features and behavior needs verification; the growth of model scale leads to computational cost bottlenecks; interpretations rely on researchers' prior assumptions, which has the risk of circular reasoning.

The dual-perspective methodology integrates multi-disciplinary tools to provide a comprehensive understanding of complex AI systems. The integration of biological topology and activation space geometry not only provides technical tools but also brings interdisciplinary thinking. This framework provides an exploration direction for AI interpretability researchers and developers, and is expected to promote the construction of safer, more controllable, and more interpretable AI systems.