LLM Synthetic Data and Causal Inference: A Hybrid Method to Preserve Causal Structure Integrity

This project proposes a hybrid synthetic data method combining large language models (LLMs) and traditional generative models, aiming to address the data sharing challenges in causal inference while maintaining the accuracy of key causal quantities such as the average treatment effect (ATE), providing a new direction for data privacy protection and collaborative research in the field of causal inference.

In fields such as medicine, social sciences, and policy evaluation, causal inference is a core method for understanding intervention effects. However, real-world data has two major pain points: first, data is difficult to share due to privacy and ethical constraints; second, observational data often has positivity violations. Existing synthetic methods (e.g., GAN, simple LLM generation) can replicate predictive statistical features but fail to preserve key causal structures (especially ATE).

The core of the hybrid method is separating the covariate distribution and causal mechanism processing: using CTGAN or LLM-based GReaT to generate synthetic covariates, then simulating the conditional distributions of treatment variables and outcome variables through fitted models, which not only preserves the covariate structure but also maintains the causal mechanism.

The project provides complete experimental code, with a process including four stages: data generation, synthetic data generation, causal inference, and quality evaluation. Results show that the hybrid method significantly outperforms fully synthetic methods in maintaining ATE estimation accuracy, indicating that generative models that only pursue predictive accuracy are insufficient for causal inference applications.

Positivity is one of the basic assumptions of causal inference. This project creatively solves this problem by generating synthetic samples from seed data and strategically filling sparse regions to build an enhanced dataset that meets the positivity assumption.

Against the backdrop of strict data privacy regulations, generating synthetic data that preserves causal structures allows research institutions to support cross-institutional collaborative research without exposing sensitive raw data.

This project provides methodological guidance for the application of synthetic data in causal inference. The core insight is: in causal application scenarios, the design of generative models must explicitly consider causal structure preservation, rather than only focusing on predictive accuracy.