DiffuMural: Restoring Dunhuang Murals with Multi-scale Diffusion
Large-scale pre-trained diffusion models have produced excellent results in
the field of conditional image generation. However, restoration of ancient
murals, as an important downstream task in this field, poses significant
challenges to diffusion model-based restoration methods due to its large
defective area and scarce training samples. Conditional restoration tasks are
more concerned with whether the restored part meets the aesthetic standards of
mural restoration in terms of overall style and seam detail, and such metrics
for evaluating heuristic image complements are lacking in current research. We
therefore propose DiffuMural, a combined Multi-scale convergence and
Collaborative Diffusion mechanism with ControlNet and cyclic consistency loss
to optimise the matching between the generated images and the conditional
control. DiffuMural demonstrates outstanding capabilities in mural restoration,
leveraging training data from 23 large-scale Dunhuang murals that exhibit
consistent visual aesthetics. The model excels in restoring intricate details,
achieving a coherent overall appearance, and addressing the unique challenges
posed by incomplete murals lacking factual grounding. Our evaluation framework
incorporates four key metrics to quantitatively assess incomplete murals:
factual accuracy, textural detail, contextual semantics, and holistic visual
coherence. Furthermore, we integrate humanistic value assessments to ensure the
restored murals retain their cultural and artistic significance. Extensive
experiments validate that our method outperforms state-of-the-art (SOTA)
approaches in both qualitative and quantitative metrics.
67fe515cfd3df6c5de11c9f2
Puyu Han
67fe515cfd3df6c5de11c9f3
Jiaju Kang
67fe515cfd3df6c5de11c9f4
Yuhang Pan
67fe515cfd3df6c5de11c9f5
Erting Pan
67fe515cfd3df6c5de11c9f6
Zeyu Zhang
claimed_verified
2025-04-15T16:46:58.052Z
64ec877bb93654d4ca5c92e9
https://cdn-avatars.huggingface.co/v1/production/uploads/64ec877bb93654d4ca5c92e9/GvHk_KSdE9Rhnk_o-NaZX.jpeg
Zeyu Zhang
user
SteveZeyuZhang
67fe515cfd3df6c5de11c9f7
Qunchao Jin
67fe515cfd3df6c5de11c9f8
Juntao Jiang
67fe515cfd3df6c5de11c9f9
Zhichen Liu
67fe515cfd3df6c5de11c9fa
Luqi Gong
DiffuMural: Restoring Dunhuang Murals with Multi-scale Diffusion
Large-scale pre-trained diffusion models have produced excellent results in
the field of conditional image generation. However, restoration of ancient
murals, as an important downstream task in this field, poses significant
challenges to diffusion model-based restoration methods due to its large
defective area and scarce training samples. Conditional restoration tasks are
more concerned with whether the restored part meets the aesthetic standards of
mural restoration in terms of overall style and seam detail, and such metrics
for evaluating heuristic image complements are lacking in current research. We
therefore propose DiffuMural, a combined Multi-scale convergence and
Collaborative Diffusion mechanism with ControlNet and cyclic consistency loss
to optimise the matching between the generated images and the conditional
control. DiffuMural demonstrates outstanding capabilities in mural restoration,
leveraging training data from 23 large-scale Dunhuang murals that exhibit
consistent visual aesthetics. The model excels in restoring intricate details,
achieving a coherent overall appearance, and addressing the unique challenges
posed by incomplete murals lacking factual grounding. Our evaluation framework
incorporates four key metrics to quantitatively assess incomplete murals:
factual accuracy, textural detail, contextual semantics, and holistic visual
coherence. Furthermore, we integrate humanistic value assessments to ensure the
restored murals retain their cultural and artistic significance. Extensive
experiments validate that our method outperforms state-of-the-art (SOTA)
approaches in both qualitative and quantitative metrics.
DiffuMural, a multi-scale diffusion model with ControlNet and cyclic consistency loss, effectively restores ancient murals by addressing large defects and scarce training data, outperforming existing methods.
diffusion models conditional image generation mural restoration Multi-scale convergence Collaborative Diffusion ControlNet cyclic consistency loss factual accuracy textural detail contextual semantics holistic visual coherence