DreamCar: Leveraging Car-specific Prior for in-the-wild 3D Car
  Reconstruction
Self-driving industries usually employ professional artists to build
exquisite 3D cars. However, it is expensive to craft large-scale digital
assets. Since there are already numerous datasets available that contain a vast
number of images of cars, we focus on reconstructing high-quality 3D car models
from these datasets. However, these datasets only contain one side of cars in
the forward-moving scene. We try to use the existing generative models to
provide more supervision information, but they struggle to generalize well in
cars since they are trained on synthetic datasets not car-specific. In
addition, The reconstructed 3D car texture misaligns due to a large error in
camera pose estimation when dealing with in-the-wild images. These restrictions
make it challenging for previous methods to reconstruct complete 3D cars. To
address these problems, we propose a novel method, named DreamCar, which can
reconstruct high-quality 3D cars given a few images even a single image. To
generalize the generative model, we collect a car dataset, named Car360, with
over 5,600 vehicles. With this dataset, we make the generative model more
robust to cars. We use this generative prior specific to the car to guide its
reconstruction via Score Distillation Sampling. To further complement the
supervision information, we utilize the geometric and appearance symmetry of
cars. Finally, we propose a pose optimization method that rectifies poses to
tackle texture misalignment. Extensive experiments demonstrate that our method
significantly outperforms existing methods in reconstructing high-quality 3D
cars. https://xiaobiaodu.github.io/dreamcar-project/{Our code is
available.}
66a1b1438f0d2327e0a19314
Xiaobiao Du
claimed_verified
2024-07-25T07:38:30.366Z
64ad086c5d48838462e2eee1
/avatars/735b29d01bae05599e7d5ce61b223153.svg
Du
user
xiaobiaodu
66a1b1438f0d2327e0a19315
Haiyang Sun
admin_assigned
2024-07-25T12:25:59.881Z
639cc617b0496dc19b6f326f
/avatars/0bfabcb1f973716b604c9d733e499242.svg
Haiyang Sun
user
haiyangsun
66a1b1438f0d2327e0a19316
Ming Lu
66a1b1438f0d2327e0a19317
Tianqing Zhu
66a1b1438f0d2327e0a19318
Xin Yu
DreamCar: Leveraging Car-specific Prior for in-the-wild 3D Car
  Reconstruction
Self-driving industries usually employ professional artists to build
exquisite 3D cars. However, it is expensive to craft large-scale digital
assets. Since there are already numerous datasets available that contain a vast
number of images of cars, we focus on reconstructing high-quality 3D car models
from these datasets. However, these datasets only contain one side of cars in
the forward-moving scene. We try to use the existing generative models to
provide more supervision information, but they struggle to generalize well in
cars since they are trained on synthetic datasets not car-specific. In
addition, The reconstructed 3D car texture misaligns due to a large error in
camera pose estimation when dealing with in-the-wild images. These restrictions
make it challenging for previous methods to reconstruct complete 3D cars. To
address these problems, we propose a novel method, named DreamCar, which can
reconstruct high-quality 3D cars given a few images even a single image. To
generalize the generative model, we collect a car dataset, named Car360, with
over 5,600 vehicles. With this dataset, we make the generative model more
robust to cars. We use this generative prior specific to the car to guide its
reconstruction via Score Distillation Sampling. To further complement the
supervision information, we utilize the geometric and appearance symmetry of
cars. Finally, we propose a pose optimization method that rectifies poses to
tackle texture misalignment. Extensive experiments demonstrate that our method
significantly outperforms existing methods in reconstructing high-quality 3D
cars. https://xiaobiaodu.github.io/dreamcar-project/{Our code is
available.}
The DreamCar method reconstructs high-quality 3D car models from limited images using a car-specific generative model, Score Distillation Sampling, and pose optimization to address texture misalignment.
generative models Score Distillation Sampling pose optimization 3D car models Car360 symmetry