DP²O-SR: Direct Perceptual Preference Optimization for Real-World Image Super-Resolution
Rongyuan Wu, Lingchen Sun, Zhengqiang ZHANG, Shihao Wang, Tianhe Wu, Qiaosi Yi, Shuai Li, Lei Zhang
Real-world image super-resolution (Real-ISR) benefits from pre-trained text-to-image diffusion models, which can generate rich details. However, these models exhibit inherent stochasticity, where different noise inputs yield results of varying perceptual quality. This stochasticity actually widens the perceptual quality *range* of outputs, which motivates us to develop **D**irect **P**erceptual **P**reference **O**ptimization for Real-I**SR** (**DP²O-SR**). We first define a Combined Perceptual Score (CPS) to quantify image perceptual quality preference, then employ Direct Preference Optimization (DPO) to fine-tune the pre-trained Real-ISR model towards generating outputs with higher CPS scores. While simply increasing the number of noise samples could improve DPO performance, this is computationally expensive. We therefore propose to extract more training signals from a limited group of candidate outputs. Instead of forming only one preference pair (best and worst samples) per input, we generate multiple pairs by selecting higher- and lower-scoring samples from the same group, significantly increasing the DPO training efficacy. In addition, considering the unequal importance of these pairs, we introduce Hierarchical Significance Weighting (HSW) to weight the DPO loss based on intra-pair significance (CPS difference within the pair) and inter-group learning potential (CPS variance of the source candidate group), prioritizing more informative pairs and maximizing DPO learning from available samples. Experiments show that **DP²O-SR** significantly enhances the perceptual quality over the diffusion-based Real-ISR models, validated by quantitative metrics and visualization comparison. The code, model, and data will be made publicly available.