Phantoms and Disclosures: A Causal Inference Framework for Synthetic Data Auditing

Original Article Information

• Original Author/Team: Privacy Protection and Data Security Research Team
• Source Platform: arXiv
• Original Title: Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data
• Original Link: http://arxiv.org/abs/2606.16952v1
• Release Time: 2026-06-15

Core Insights The research team proposes a causal inference-based privacy auditing framework for synthetic data. By distinguishing between "real disclosures" and "phantom disclosures" and combining statistical hypothesis testing, it achieves privacy leak detection without model access, canary insertion, or reference model training, and provides a tighter lower bound on privacy leaks than existing methods.

The rapid development of generative AI and large language models has spurred huge demand for synthetic data. As a privacy-preserving alternative to sensitive real data, it is widely used in healthcare, finance, and other fields. However, there is a high quality vs. privacy protection paradox: high-quality synthetic data needs to retain the statistical properties, feature correlations, and downstream task support capabilities of real data, but this easily increases the risk of memorizing sensitive information.

Privacy disclosure risks include:

1. Verbatim copying: Synthetic data contains records identical to those in the training data
2. Approximate reproduction: Synthetic data is highly similar to training records, allowing inference of sensitive information
3. Attribute disclosure: Leaking specific attributes of individuals in the training set
4. Membership inference: Determining whether a record is in the training set through synthetic data

Existing methods have obvious shortcomings:

• Canary insertion method: Requires modifying training data, only detects memorization of specific records, and easily introduces bias
• Shadow model method: Extremely high computational cost, differences between shadow and target models make it hard to scale
• Reference model method: Requires additional reference model training, selection affects result reliability, and cannot handle distribution shifts
• Model access dependency: Needs access to model parameters/gradients, not suitable for black-box API scenarios, and may leak intellectual property

The core of the framework lies in distinguishing two types of disclosures:

• Real disclosure: The system directly copies/approximately reproduces sensitive information from the training data, with a causal relationship (e.g., training data has "Zhang San, ID number 123456", and synthetic data contains the same record)
• Phantom disclosure: Synthetic data is similar to a record but originates from statistical properties rather than memorization (e.g., training data has "Li Si, 30 years old, income 50,000", and synthetic data contains "Wang Wu, 30 years old, income 50,000")

Importance of distinction: Avoid false positives, accurately assess privacy risks, and guide developers to make targeted improvements.

Data Partitioning Strategy

Split input data into a training set (for model generation) and a holdout set (same distribution but not trained). Assumption: Similarity between synthetic data and the training set may indicate memorization, while similarity with the holdout set may be a coincidence. Partition methods include random, stratified, and time-based partitioning.

Statistical Hypothesis Testing

• Zero-learning baseline: Test whether the similarity between synthetic data and the training set is higher than that of a zero-learning model (H0: ≤ expected similarity; H1: > expected similarity)
• Differential privacy baseline: Test whether actual disclosure complies with the declared DP budget (H0: ≤ declared boundary; H1: > declared boundary)
• Test methods: Kolmogorov-Smirnov test, Mann-Whitney U test, permutation test

Membership Inference Attack Perspective

Quantify the membership inference success rate by comparing the similarity distributions of the training set and holdout set (the higher the success rate, the more severe the privacy leak).

The framework has the following advantages:

• No model access: Only requires synthetic data, suitable for black-box APIs, third-party data, and compliance audit scenarios
• No canary insertion: Does not modify training data, suitable for deployed systems, and assesses overall risk
• No reference model training: Reduces computational cost by orders of magnitude and avoids subjectivity in reference model selection
• Model agnostic: Applicable to various synthetic mechanisms such as GAN, VAE, diffusion models, LLM, tabular/time-series data
• Computational efficiency: Similarity calculation O(|synthetic data| × |training data|), statistical testing O(|synthetic data|), far lower than shadow model methods

Experimental Setup

• Datasets: UCI tabular data, text data (news/social posts), time-series data (sensor/finance)
• Synthetic methods: Gaussian mixture model, Bayesian network, GAN, VAE, diffusion model, GPT/Llama series
• Comparison baselines: Canary method, shadow model, traditional membership inference attack

Core Results

• Disclosure detection capability: High recall rate (detects most real disclosures), low false positive rate (distinguishes real vs. phantom disclosures)
• Method comparison:

  Method	Computational Cost	Model Access	False Positive Rate	Applicable Scope
  Our framework	Low	No	Low	Universal
  Canary	Low	No	High	Specific records
  Shadow model	Extremely high	Yes	Medium	White-box
  Traditional MIA	Medium	Yes	Medium	White-box

• Privacy lower bound: Provides a tighter lower bound on privacy leaks than existing methods

Case Analysis

• Medical data: Real disclosures of rare disease patients were detected in GAN-synthesized patient records (caused by overfitting)
• LLM text: GPT-generated news contained verbatim copies of training data, requiring distinction from phantom disclosures

Application Recommendations

• Pre-release audit: Data partitioning → generate synthetic data → run audit → assess risk
• Continuous monitoring: Regular sampling audit, set disclosure threshold alerts, trend analysis
• Compliance support: Assist GDPR (anonymization assessment), HIPAA (medical data de-identification), CCPA (consumer privacy audit)

Limitations

1. Similarity metric selection affects results
2. Insufficient statistical test power for small datasets
3. Distinguishing real vs. phantom disclosures is based on statistical methods, not deterministic

Future Directions

1. Develop data-adaptive similarity metrics
2. Extend to multi-dimensional leaks such as attribute and relationship leaks
3. Establish formal theoretical guarantees for the framework
4. Develop real-time auditing methods for streaming data