STRIDE: Activating Spatial Data Attribution with 13x Speed Improvement

Core Insights

STRIDE is a training data attribution tool that increases attribution speed by 13x using spatial modeling activation and sparse recovery techniques, providing efficient solutions for LLM scenarios such as data selection and contamination detection.

Source Information

• Paper Title: STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations
• Publication Platform: arXiv
• Publication Date: June 3, 2026
• Original Link: http://arxiv.org/abs/2606.05165v1

Importance of Training Data Attribution

Training Data Attribution (TDA) is a core issue in machine learning interpretability, related to understanding model behavior, ensuring data quality, and complying with regulations.

Dilemma of the Gold Standard

Causal intervention (remove samples → retrain → compare predictions) is the gold standard for TDA, but its computational cost is extremely high for LLMs (retraining billions of parameters costs millions of dollars).

Limitations of Existing Methods

• Gradient Tracking: Relies on gradient calculations in the parameter space, with high overhead and dependence on local linear approximation.
• Influence Functions: Require calculation of the Hessian matrix, with extremely high storage and computational costs. Both are limited by the high dimensionality of the parameter space.

Paradigm Shift: From Parameter to Activation Space

The core insight of STRIDE is to shift TDA from the parameter space to the activation space—activation dimensions are much lower than parameter dimensions and directly reflect model behavior.

Technical Framework

1. Guiding Operator: A lightweight linear transformation that simulates model behavior after training on a specific subset of data and matches the output distribution.
2. Subset Perturbation: Sample training subsets → learn guiding operators → decompose changes in test predictions.
3. Sparse Recovery: Based on the idea of compressed sensing, assuming that a single prediction is only affected by a few samples, and using L1 regularization for efficient solution.

Performance

• Speed: 13x faster than state-of-the-art methods.
• Accuracy: Reaches or exceeds existing methods on multiple datasets.
• Scalability: Applicable to LLMs with billions of parameters.

Method Comparison

Method	Speed	Accuracy	Scalability
Retraining	Extremely slow	Highest	Not feasible
Gradient Tracking	Slow	Medium	Limited
Influence Functions	Medium	Medium	Limited
STRIDE	Fast	High	Excellent

Ablation Experiments

• Spatial modeling activation is the main source of speed improvement.
• Sparse recovery significantly reduces computational costs.
• The design of lightweight guiding operators is key to efficiency.

Data Selection

Identify the most important training samples for specific tasks, achieving comparable performance with less data.

Data Contamination Detection

Locate mislabeled or contaminated samples that have abnormal impacts on model behavior.

Qualitative Analysis

Trace predictions to specific training samples, revealing unexpected associations and biases learned by the model.

Current Limitations

• Approximation error: Cannot completely replace retraining.
• Sparse assumption: May not hold for some highly integrated tasks.
• Computational requirements: Still require significant resources.

Future Directions

• Hierarchical attribution: Attribute to specific parts of samples.
• Temporal modeling: Track dynamic changes in data impact during training.
• Multimodal extension: Support multimodal models such as vision-language.
• Real-time applications: Develop online real-time attribution systems.

Applicable Scenarios

1. Model debugging: Understand abnormal behavior.
2. Data auditing: Verify data quality and appropriateness.
3. Compliance requirements: Meet data traceability regulations.
4. Data optimization: Remove low-quality or harmful samples.

Implementation Recommendations

• Precompute guiding operators: Pre-store for key subsets.
• Hierarchical attribution: First batches, then samples.
• Combine with manual review: Use STRIDE results to guide priorities.