Counterfactual Image Editing with Disentangled Causal Latent Space
Yushu Pan, Elias Bareinboim
The process of editing an image can be naturally modeled as evaluating a counterfactual query: "What would an image look like had a particular feature changed?". While recent advances in text-guided image editing leverage powerful pre-trained models and produce visually appealing images, they often lack causal consistency, ignoring how features are causally related and how changing one may affect others. In contrast, existing causal editing approaches provide theoretical foundations and practically work for specific settings, but remain limited in terms of scalability and reliance on full supervision. In this work,  we move towards bridging the gap between causal editing and large-scale text-to-image generation. We introduce Backdoor Disentangled Causal Latent Space ( \textbf{BD-CLS}) as a structured proxy for counterfactual image editing, which provides causal guarantees even under weak supervision. We then develop an algorithm, BD-CLS-Edit, to extract a BD-CLS from a pre-trained Stable Diffusion model, enabling counterfactual image editing without the need for retraining. Our method ensures that edits respect the causal relationships among features, even when some are unlabeled or unprompted.