Deconstructing GPT-2's Grammar Circuits: A Causal Analysis of Part-of-Speech Encoding Mechanisms in Large Language Models

This study, through the open-source GPT2_MI project and combining linear probing, causal activation patching, and sparse autoencoder (SAE) techniques, systematically reveals the part-of-speech (POS) encoding mechanism inside the GPT-2 Small model. It provides interpretable mechanistic insights into the grammatical processing capabilities of large language models (LLMs). This article will cover research background, technical approach, core findings, practical implications, and other content across different floors.

Large language models (LLMs) demonstrate impressive grammatical correctness in natural language processing tasks, but the representation and processing mechanisms of internal grammatical information have long been a black-box problem. Traditional evaluations only focus on input and output, failing to reveal internal working mechanisms. Part-of-speech (POS) is key to understanding sentence structure—if LLMs truly "understand" grammar, they must encode POS information internally. The GPT2_MI project is based on this hypothesis and attempts to locate and analyze the "grammar circuits" in GPT-2 Small through causal inference.

1. Linear Probing

Extract hidden states from different layers of the model, train a linear classifier to predict POS tags, track the flow trajectory of POS information, and find that it is highly concentrated in specific layers.

2. Causal Activation Patching

Artificially replace the activation values of a layer/group of neurons with those corresponding to a specific POS, observe output changes, and verify the causal importance of neurons in POS processing.

3. Sparse Autoencoder (SAE) Feature Decomposition

Decompose high-dimensional entangled hidden states into sparse linear combinations, identify features sensitive to specific POS, and obtain interpretable neuron combinations with clear semantics.

Layer Specificity

POS information is mainly encoded and processed in the middle layers (layers 4-8). Shallow layers focus on lexicon and local context, while deep layers handle abstract semantics and discourse information.

Distributed Representation

POS information is distributed across multiple neuron activation patterns. Each POS corresponds to a specific activation pattern of a set of feature vectors, endowing the model with generalization ability and fault tolerance.

Role of Attention Heads

Attention heads that specifically focus on POS information are identified; they tend to associate nouns with modifiers, verbs with subjects/objects, forming local syntactic dependency structures.

• Model Interpretability: Locate grammar circuits and trace abnormal components responsible for the model's grammatical errors.
• Model Editing and Control: Intervene in specific neurons to correct model behavior, e.g., adjust the frequency of POS usage.
• Education and Popular Science: Provide cases for AI education, demonstrating that models are composed of analyzable components.
• Cross-Language Transfer: The methodology can be transferred to other languages and larger models, providing a framework for multilingual grammar research.

Contributions: Demonstrate a systematic method for studying the internal grammatical mechanisms of language models. The combination of linear probing, causal patching, and SAE provides a reusable technical paradigm for mechanistic interpretability.

Limitations:

1. Focuses on GPT-2 Small (124 million parameters); grammar circuits in larger models may be more complex and dispersed;
2. Mainly focuses on POS; analysis of complex syntactic structures (e.g., clause nesting) needs further exploration.

• Apply to multilingual models to explore commonalities in grammar circuits across different languages;
• Investigate whether grammatical processing mechanisms in large-scale models exhibit emergent new features;
• Use the findings to design more controllable and interpretable next-generation language models.

This project provides a technical reference for understanding the working mechanisms of LLMs and establishes a research paradigm using rigorous scientific methods to demystify the AI black box.