Instant Facial Gaussians Translator for Relightable and Interactable
  Facial Rendering
We propose GauFace, a novel Gaussian Splatting representation, tailored for
efficient animation and rendering of physically-based facial assets. Leveraging
strong geometric priors and constrained optimization, GauFace ensures a neat
and structured Gaussian representation, delivering high fidelity and real-time
facial interaction of 30fps@1440p on a Snapdragon 8 Gen 2 mobile platform.
  Then, we introduce TransGS, a diffusion transformer that instantly translates
physically-based facial assets into the corresponding GauFace representations.
Specifically, we adopt a patch-based pipeline to handle the vast number of
Gaussians effectively. We also introduce a novel pixel-aligned sampling scheme
with UV positional encoding to ensure the throughput and rendering quality of
GauFace assets generated by our TransGS. Once trained, TransGS can instantly
translate facial assets with lighting conditions to GauFace representation,
With the rich conditioning modalities, it also enables editing and animation
capabilities reminiscent of traditional CG pipelines.
  We conduct extensive evaluations and user studies, compared to traditional
offline and online renderers, as well as recent neural rendering methods, which
demonstrate the superior performance of our approach for facial asset
rendering. We also showcase diverse immersive applications of facial assets
using our TransGS approach and GauFace representation, across various platforms
like PCs, phones and even VR headsets.
66e258a25f29feaa90934c4d
Dafei Qin
admin_assigned
2024-09-12T10:25:39.572Z
65b8c8b3fc836d9cd5b8a6ec
/avatars/9c2490aebcca7b16396302f4f96acdfd.svg
Dafei Qin
user
dafeiqin
66e258a25f29feaa90934c4e
Hongyang Lin
66e258a25f29feaa90934c4f
Qixuan Zhang
admin_assigned
2024-09-12T10:25:54.414Z
661d10eee7b0ab12bc6721d1
/avatars/a35c91825baa90d4b0703a1e06ccc5f4.svg
Qixuan Zhang
user
Haaribo
66e258a25f29feaa90934c50
Kaichun Qiao
admin_assigned
2024-09-12T10:25:59.975Z
64c8c537cb2f1bf0e7c3e99d
/avatars/ce8df2802840120022ec8ebaea4f525e.svg
Kaichun Qiao
user
DolphinQiao
66e258a25f29feaa90934c51
Longwen Zhang
claimed_verified
2025-07-31T08:17:25.217Z
636d12455aaed143cd665607
https://cdn-avatars.huggingface.co/v1/production/uploads/1679399015950-636d12455aaed143cd665607.png
ZLW
user
ZarkLngeW
66e258a25f29feaa90934c52
Zijun Zhao
66e258a25f29feaa90934c53
Jun Saito
66e258a25f29feaa90934c54
Jingyi Yu
admin_assigned
2024-09-12T10:27:04.227Z
63ffcec6e7767a89533b8f80
/avatars/b9e8673943741c2e7b80b15a061bb951.svg
JINGYI Yu
user
Daluuu
66e258a25f29feaa90934c55
Lan Xu
66e258a25f29feaa90934c56
Taku Komura
Instant Facial Gaussians Translator for Relightable and Interactable
  Facial Rendering
We propose GauFace, a novel Gaussian Splatting representation, tailored for
efficient animation and rendering of physically-based facial assets. Leveraging
strong geometric priors and constrained optimization, GauFace ensures a neat
and structured Gaussian representation, delivering high fidelity and real-time
facial interaction of 30fps@1440p on a Snapdragon 8 Gen 2 mobile platform.
  Then, we introduce TransGS, a diffusion transformer that instantly translates
physically-based facial assets into the corresponding GauFace representations.
Specifically, we adopt a patch-based pipeline to handle the vast number of
Gaussians effectively. We also introduce a novel pixel-aligned sampling scheme
with UV positional encoding to ensure the throughput and rendering quality of
GauFace assets generated by our TransGS. Once trained, TransGS can instantly
translate facial assets with lighting conditions to GauFace representation,
With the rich conditioning modalities, it also enables editing and animation
capabilities reminiscent of traditional CG pipelines.
  We conduct extensive evaluations and user studies, compared to traditional
offline and online renderers, as well as recent neural rendering methods, which
demonstrate the superior performance of our approach for facial asset
rendering. We also showcase diverse immersive applications of facial assets
using our TransGS approach and GauFace representation, across various platforms
like PCs, phones and even VR headsets.
GauFace, a Gaussian Splatting representation, and TransGS, a diffusion transformer, enable efficient and high-fidelity rendering of facial assets with real-time performance across multiple platforms.
Gaussian Splatting constrained optimization Gaussian representation diffusion transformer patch-based pipeline UV positional encoding neural rendering methods immersive applications