FM-CGM: Causal Generative Modeling and Counterfactual Reasoning Using Foundation Models

This article introduces the FM-CGM framework published on arXiv on May 22, 2026 (original paper link: http://arxiv.org/abs/2605.23861v1). Its core is combining large reasoning models with diffusion models to achieve zero-shot causal discovery, intervention, and counterfactual image generation, completing causal reasoning without retraining. This framework provides an innovative approach for integrating causal AI and foundation models.

Causal reasoning is key to AI moving toward higher intelligence, and counterfactual reasoning is the core of human cognition. Existing causal generative models have problems such as high training costs, limited generalization ability, lack of a unified framework, and inability to leverage the zero-shot reasoning capabilities of pre-trained foundation models. FM-CGM is an innovative solution targeting these pain points.

FM-CGM includes three core components:

1. Concept Extractor (extracts semantic concepts from images as nodes of the causal graph);
2. Concept Operator (uses large reasoning models to infer causal relationships and simulate interventions);
3. Counterfactual Generator (uses diffusion models to generate images after intervention). In addition, the Causal Semantic Guidance (CSG) mechanism ensures the propagation of semantic interventions and the stability of unchanged regions through cross-attention, solving the consistency problem in counterfactual generation.

FM-CGM relies on pre-trained foundation models and can complete the following without specific training:

1. Causal discovery (identifies latent causal structures from observational data);
2. Intervention simulation (predicts results after variable intervention);
3. Counterfactual generation (generates "what if" images). For example, it can turn a sunny street scene into a cloudy one, adjust related elements like light and tone, while keeping unrelated elements like buildings unchanged.

Experiments show that FM-CGM is effective in multiple tasks:

1. The accuracy of causal structure identification is significantly higher than statistical correlation baselines;
2. Counterfactual images perform well in semantic consistency, visual quality, and fine-grained control (validated via FID, CLIP scores, and human evaluation);
3. Compared to models requiring specialized training, it has higher flexibility and lower deployment costs.

Applications of FM-CGM include:

1. Data augmentation (generates edge case data in medical imaging and autonomous driving fields);
2. Model interpretability (understands decision boundaries through counterfactual examples);
3. Creative design (quickly explores different visual styles);
4. Scientific discovery (low-cost exploration of causal hypotheses, such as climate modeling and epidemiology).

Limitations of FM-CGM: depends on the common sense boundaries of foundation models, reasoning accuracy is affected by the capabilities of large models, and detail control needs improvement. However, it successfully validates the value of compositional AI, proves that the zero-shot capabilities of pre-trained models can be used for complex causal reasoning, pushes AI from correlation learning to causal understanding, and opens up new directions for causal AI.