FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animations
Sketch animations offer a powerful medium for visual storytelling, from
simple flip-book doodles to professional studio productions. While traditional
animation requires teams of skilled artists to draw key frames and in-between
frames, existing automation attempts still demand significant artistic effort
through precise motion paths or keyframe specification. We present FlipSketch,
a system that brings back the magic of flip-book animation -- just draw your
idea and describe how you want it to move! Our approach harnesses motion priors
from text-to-video diffusion models, adapting them to generate sketch
animations through three key innovations: (i) fine-tuning for sketch-style
frame generation, (ii) a reference frame mechanism that preserves visual
integrity of input sketch through noise refinement, and (iii) a dual-attention
composition that enables fluid motion without losing visual consistency. Unlike
constrained vector animations, our raster frames support dynamic sketch
transformations, capturing the expressive freedom of traditional animation. The
result is an intuitive system that makes sketch animation as simple as doodling
and describing, while maintaining the artistic essence of hand-drawn animation.
673df3308f9f9408fe5b8d41
Hmrishav Bandyopadhyay
claimed_verified
2024-11-20T15:31:23.868Z
638c81fa61eb51017518fa31
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/f0eCrzBrxz7Y9n25WkZ2v.png
Hmrishav Bandyopadhyay
user
Hmrishav
673df3308f9f9408fe5b8d42
Yi-Zhe Song
FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animations
Sketch animations offer a powerful medium for visual storytelling, from
simple flip-book doodles to professional studio productions. While traditional
animation requires teams of skilled artists to draw key frames and in-between
frames, existing automation attempts still demand significant artistic effort
through precise motion paths or keyframe specification. We present FlipSketch,
a system that brings back the magic of flip-book animation -- just draw your
idea and describe how you want it to move! Our approach harnesses motion priors
from text-to-video diffusion models, adapting them to generate sketch
animations through three key innovations: (i) fine-tuning for sketch-style
frame generation, (ii) a reference frame mechanism that preserves visual
integrity of input sketch through noise refinement, and (iii) a dual-attention
composition that enables fluid motion without losing visual consistency. Unlike
constrained vector animations, our raster frames support dynamic sketch
transformations, capturing the expressive freedom of traditional animation. The
result is an intuitive system that makes sketch animation as simple as doodling
and describing, while maintaining the artistic essence of hand-drawn animation.
FlipSketch generates sketch animations by leveraging text-to-video diffusion models with fine-tuning, noise refinement, and dual-attention composition to create fluid and visually consistent animations from simple sketches and descriptions.
motion priors text-to-video diffusion models fine-tuning reference frame mechanism noise refinement dual-attention composition raster frames dynamic sketch transformations fluid motion visual consistency