TopoMIA: Research on Topology-Aware Membership Inference Attacks Against Black-Box Large Reasoning Models

TopoMIA is a security study targeting black-box large reasoning models, proposing a topology-aware membership inference attack method and revealing potential privacy protection risks of large reasoning models. By analyzing the topological structure differences in the model's chain of thought (distinct reasoning path characteristics between training samples and non-training samples), this study achieves effective attacks in black-box settings, providing new perspectives and defense directions for the AI security field.

Black-Box Characteristics of Large Reasoning Models

Large reasoning models (e.g., OpenAI o1, DeepSeek-R1) are served as black-box APIs, only outputting results and chains of thought without access to internal states. While this protects intellectual property, it introduces security risks.

Definition of Membership Inference Attacks

Membership Inference Attacks (MIA) aim to determine whether a sample belongs to a model's training set, which is particularly dangerous for models containing sensitive data (e.g., private data, trade secrets).

Limitations of Traditional Methods

Traditional MIA relies on output confidence or loss values, but the chain-of-thought output of black-box reasoning models provides an additional dimension of information, which traditional methods struggle to handle.

Topology-Aware Method

The core of TopoMIA is analyzing the topological features of the reasoning process (chain-of-thought expansion method, step organization, logical branches). It was found that the reasoning paths of training samples are more direct and confident, while those of unfamiliar samples are longer and have more branches.

BookReasoning-Chinese Dataset

A Chinese dataset specifically designed to evaluate the security of reasoning models is introduced to test the cross-language capability of attacks, filling the gap in high-quality Chinese AI security datasets.

Attack Flow

1. Feature Extraction: Extract topological features from the chain of thought (reasoning depth, number of branches, backtracking frequency);
2. Topology Analysis: Model the chain of thought as a graph structure (nodes = steps, edges = logical dependencies) and analyze structural differences;
3. Classification Decision: Train a binary classifier using topological features to determine if a sample is a member.

Experimental Results

TopoMIA achieves a significant success rate on mainstream reasoning models and is entirely based on black-box API queries, closely simulating real-world attack scenarios.

Risks of Chain of Thought

While chain of thought improves interpretability, it leaks additional information, requiring a balance between transparency and security.

Privacy Vulnerabilities of Black-Box Models

Even with black-box deployment, training data information may still be leaked through behavioral patterns, warning organizations that train models with sensitive data.

Defense Strategy Recommendations

• Perturb/abstract the chain of thought to reduce information leakage;
• Adopt differential privacy to protect training data;
• Develop mechanisms to detect and block MIA queries.

Academic Frontier

As a submission to ACM CCS 2026, it represents the frontier of security research.

Open-Source and Dataset

Experimental code and evaluation scripts are open-sourced, and the BookReasoning-Chinese dataset is released to facilitate reproducibility and further research in the field.

Attack Expansion

Explore more refined topological features, integrate side-channel information, and expand to multimodal reasoning models.

Defense Optimization

Develop defense solutions that balance privacy and performance.

Applications in High-Risk Domains

Security research in fields such as healthcare, finance, and law needs to be advanced simultaneously to address potential serious consequences.