AlteredAvatar: Stylizing Dynamic 3D Avatars with Fast Style Adaptation
This paper presents a method that can quickly adapt dynamic 3D avatars to
arbitrary text descriptions of novel styles. Among existing approaches for
avatar stylization, direct optimization methods can produce excellent results
for arbitrary styles but they are unpleasantly slow. Furthermore, they require
redoing the optimization process from scratch for every new input. Fast
approximation methods using feed-forward networks trained on a large dataset of
style images can generate results for new inputs quickly, but tend not to
generalize well to novel styles and fall short in quality. We therefore
investigate a new approach, AlteredAvatar, that combines those two approaches
using the meta-learning framework. In the inner loop, the model learns to
optimize to match a single target style well; while in the outer loop, the
model learns to stylize efficiently across many styles. After training,
AlteredAvatar learns an initialization that can quickly adapt within a small
number of update steps to a novel style, which can be given using texts, a
reference image, or a combination of both. We show that AlteredAvatar can
achieve a good balance between speed, flexibility and quality, while
maintaining consistency across a wide range of novel views and facial
expressions.
6476e1ca83d4fdaedde06ea7
Thu Nguyen-Phuoc
6476e1ca83d4fdaedde06ea8
Gabriel Schwartz
admin_assigned
2023-06-02T12:34:16.522Z
63b261b9d5f79ac67d2cace3
/avatars/fa560a2e7732cf9e4cd85db3496fd6b2.svg
Gabriel Schwartz 
user
GBielXONE02
6476e1ca83d4fdaedde06ea9
Yuting Ye
6476e1ca83d4fdaedde06eaa
Stephen Lombardi
6476e1ca83d4fdaedde06eab
Lei Xiao
AlteredAvatar: Stylizing Dynamic 3D Avatars with Fast Style Adaptation
This paper presents a method that can quickly adapt dynamic 3D avatars to
arbitrary text descriptions of novel styles. Among existing approaches for
avatar stylization, direct optimization methods can produce excellent results
for arbitrary styles but they are unpleasantly slow. Furthermore, they require
redoing the optimization process from scratch for every new input. Fast
approximation methods using feed-forward networks trained on a large dataset of
style images can generate results for new inputs quickly, but tend not to
generalize well to novel styles and fall short in quality. We therefore
investigate a new approach, AlteredAvatar, that combines those two approaches
using the meta-learning framework. In the inner loop, the model learns to
optimize to match a single target style well; while in the outer loop, the
model learns to stylize efficiently across many styles. After training,
AlteredAvatar learns an initialization that can quickly adapt within a small
number of update steps to a novel style, which can be given using texts, a
reference image, or a combination of both. We show that AlteredAvatar can
achieve a good balance between speed, flexibility and quality, while
maintaining consistency across a wide range of novel views and facial
expressions.
AlteredAvatar combines direct optimization and fast approximation methods for avatar stylization using meta-learning to achieve a balance between speed, flexibility, and quality.
meta-learning direct optimization feed-forward networks avatar stylization style images text descriptions reference images update steps novel styles consistency novel views facial expressions