MakeAnything: Harnessing Diffusion Transformers for Multi-Domain
  Procedural Sequence Generation
A hallmark of human intelligence is the ability to create complex artifacts
through structured multi-step processes. Generating procedural tutorials with
AI is a longstanding but challenging goal, facing three key obstacles: (1)
scarcity of multi-task procedural datasets, (2) maintaining logical continuity
and visual consistency between steps, and (3) generalizing across multiple
domains. To address these challenges, we propose a multi-domain dataset
covering 21 tasks with over 24,000 procedural sequences. Building upon this
foundation, we introduce MakeAnything, a framework based on the diffusion
transformer (DIT), which leverages fine-tuning to activate the in-context
capabilities of DIT for generating consistent procedural sequences. We
introduce asymmetric low-rank adaptation (LoRA) for image generation, which
balances generalization capabilities and task-specific performance by freezing
encoder parameters while adaptively tuning decoder layers. Additionally, our
ReCraft model enables image-to-process generation through spatiotemporal
consistency constraints, allowing static images to be decomposed into plausible
creation sequences. Extensive experiments demonstrate that MakeAnything
surpasses existing methods, setting new performance benchmarks for procedural
generation tasks.
67a1dc124fc394b2aa6338d5
Yiren Song
claimed_verified
2025-02-04T09:38:41.538Z
64311a95034ecbefddd141ef
/avatars/b6dc5ca373bedbaa368208517954c375.svg
Yiren Song
user
yiren98
67a1dc124fc394b2aa6338d6
Cheng Liu
claimed_verified
2025-02-07T09:58:24.249Z
6534924de778506c5b1c614c
/avatars/55417e6e8a561df06836a4ad0912080e.svg
Cheng Liu
user
lc03lc
67a1dc124fc394b2aa6338d7
Mike Zheng Shou
extracted_pending
2025-02-04T09:21:27.413Z
63a55320ce5763e06f78519c
https://cdn-avatars.huggingface.co/v1/production/uploads/1671779060549-noauth.jpeg
Mike Shou
user
mikeshou
MakeAnything: Harnessing Diffusion Transformers for Multi-Domain
  Procedural Sequence Generation
A hallmark of human intelligence is the ability to create complex artifacts
through structured multi-step processes. Generating procedural tutorials with
AI is a longstanding but challenging goal, facing three key obstacles: (1)
scarcity of multi-task procedural datasets, (2) maintaining logical continuity
and visual consistency between steps, and (3) generalizing across multiple
domains. To address these challenges, we propose a multi-domain dataset
covering 21 tasks with over 24,000 procedural sequences. Building upon this
foundation, we introduce MakeAnything, a framework based on the diffusion
transformer (DIT), which leverages fine-tuning to activate the in-context
capabilities of DIT for generating consistent procedural sequences. We
introduce asymmetric low-rank adaptation (LoRA) for image generation, which
balances generalization capabilities and task-specific performance by freezing
encoder parameters while adaptively tuning decoder layers. Additionally, our
ReCraft model enables image-to-process generation through spatiotemporal
consistency constraints, allowing static images to be decomposed into plausible
creation sequences. Extensive experiments demonstrate that MakeAnything
surpasses existing methods, setting new performance benchmarks for procedural
generation tasks.
A diffusion transformer-based framework, MakeAnything, generates coherent procedural tutorials across domains using a large dataset and asymmetric low-rank adaptation, outperforming existing methods.
diffusion transformer (DIT) fine-tuning asymmetric low-rank adaptation (LoRA) image generation ReCraft spatiotemporal consistency procedural tutorials procedural generation tasks