DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training
In this work, we propose DiT360, a DiT-based framework that performs hybrid
training on perspective and panoramic data for panoramic image generation. For
the issues of maintaining geometric fidelity and photorealism in generation
quality, we attribute the main reason to the lack of large-scale, high-quality,
real-world panoramic data, where such a data-centric view differs from prior
methods that focus on model design. Basically, DiT360 has several key modules
for inter-domain transformation and intra-domain augmentation, applied at both
the pre-VAE image level and the post-VAE token level. At the image level, we
incorporate cross-domain knowledge through perspective image guidance and
panoramic refinement, which enhance perceptual quality while regularizing
diversity and photorealism. At the token level, hybrid supervision is applied
across multiple modules, which include circular padding for boundary
continuity, yaw loss for rotational robustness, and cube loss for distortion
awareness. Extensive experiments on text-to-panorama, inpainting, and
outpainting tasks demonstrate that our method achieves better boundary
consistency and image fidelity across eleven quantitative metrics. Our code is
available at https://github.com/Insta360-Research-Team/DiT360.
68edbf46de1fee572713a874
Haoran Feng
claimed_verified
2025-10-14T07:30:40.094Z
65240d0ca801972b6eb12ed8
https://cdn-avatars.huggingface.co/v1/production/uploads/65240d0ca801972b6eb12ed8/hl2RAssBperb5JlgOIDvw.jpeg
Haoran Feng
user
fenghora
68edbf46de1fee572713a875
Dizhe Zhang
68edbf46de1fee572713a876
Xiangtai Li
68edbf46de1fee572713a877
Bo Du
68edbf46de1fee572713a878
Lu Qi
DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training
In this work, we propose DiT360, a DiT-based framework that performs hybrid
training on perspective and panoramic data for panoramic image generation. For
the issues of maintaining geometric fidelity and photorealism in generation
quality, we attribute the main reason to the lack of large-scale, high-quality,
real-world panoramic data, where such a data-centric view differs from prior
methods that focus on model design. Basically, DiT360 has several key modules
for inter-domain transformation and intra-domain augmentation, applied at both
the pre-VAE image level and the post-VAE token level. At the image level, we
incorporate cross-domain knowledge through perspective image guidance and
panoramic refinement, which enhance perceptual quality while regularizing
diversity and photorealism. At the token level, hybrid supervision is applied
across multiple modules, which include circular padding for boundary
continuity, yaw loss for rotational robustness, and cube loss for distortion
awareness. Extensive experiments on text-to-panorama, inpainting, and
outpainting tasks demonstrate that our method achieves better boundary
consistency and image fidelity across eleven quantitative metrics. Our code is
available at https://github.com/Insta360-Research-Team/DiT360.
DiT360 framework enhances panoramic image generation by hybrid training on perspective and panoramic data, incorporating cross-domain knowledge and hybrid supervision to improve boundary consistency and image fidelity.
DiT hybrid training perspective data panoramic data geometric fidelity photorealism pre-VAE post-VAE cross-domain knowledge perspective image guidance panoramic refinement perceptual quality diversity circular padding yaw loss cube loss text-to-panorama inpainting outpainting