On Fairness of Unified Multimodal Large Language Model for Image Generation
Ming Liu, Hao Chen, Jindong Wang, Liwen Wang, Bhiksha Raj, Wensheng Zhang
Unified multimodal large language models (U-MLLMs) have demonstrated impressive performance in end-to-end visual understanding and generation. However, compared to generation-only systems (e.g., Stable Diffusion), their unified architecture introduces new risks of propagating demographic stereotypes. In this paper, we benchmark several state-of-the-art U-MLLMs and show that they exhibit significant gender and race biases in their generated outputs. To diagnose the source of these biases, we propose a \emph{locate-then-fix} framework: we first audit the vision and language components—using techniques such as linear probing and controlled generation—and find that the language model appears to be a primary origin of the observed generative bias. Moreover, we observe a ``partial alignment'' phenomenon, where U-MLLMs exhibit less bias in understanding tasks yet produce substantially biased images. To address this, we introduce a novel \emph{balanced preference loss} that enforces uniform generation probabilities across demographics by leveraging a synthetically balanced dataset. Extensive experiments demonstrate that our approach significantly reduces demographic bias while preserving semantic fidelity and image quality. Our findings underscore the need for targeted debiasing strategies in unified multimodal systems and provide a practical method for mitigating bias.