GameFactory: Creating New Games with Generative Interactive Videos
Generative game engines have the potential to revolutionize game development
by autonomously creating new content and reducing manual workload. However,
existing video-based game generation methods fail to address the critical
challenge of scene generalization, limiting their applicability to existing
games with fixed styles and scenes. In this paper, we present GameFactory, a
framework focused on exploring scene generalization in game video generation.
To enable the creation of entirely new and diverse games, we leverage
pre-trained video diffusion models trained on open-domain video data. To bridge
the domain gap between open-domain priors and small-scale game dataset, we
propose a multi-phase training strategy that decouples game style learning from
action control, preserving open-domain generalization while achieving action
controllability. Using Minecraft as our data source, we release GF-Minecraft, a
high-quality and diversity action-annotated video dataset for research.
Furthermore, we extend our framework to enable autoregressive
action-controllable game video generation, allowing the production of
unlimited-length interactive game videos. Experimental results demonstrate that
GameFactory effectively generates open-domain, diverse, and action-controllable
game videos, representing a significant step forward in AI-driven game
generation. Our dataset and project page are publicly available at
https://vvictoryuki.github.io/gamefactory/.
678719ac5333dfbf8e206077
Jiwen Yu
claimed_verified
2025-01-15T08:49:01.033Z
64105a6d14215c0775dfdd14
https://cdn-avatars.huggingface.co/v1/production/uploads/64105a6d14215c0775dfdd14/-VX-cUYOLjHIg7QnWhRGG.jpeg
Jiwen Yu
user
VictorYuki
678719ac5333dfbf8e206078
Yiran Qin
claimed_verified
2025-03-21T11:49:26.310Z
658a6c1399ed106ac8c822b1
https://cdn-avatars.huggingface.co/v1/production/uploads/658a6c1399ed106ac8c822b1/Wk2KXCcK39rUvXx6mpmGD.jpeg
yiranqin
user
IranQin
678719ac5333dfbf8e206079
Xintao Wang
admin_assigned
2025-01-21T10:08:34.083Z
60e272ca6c78a8c122b12127
https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
Xintao Wang
user
Xintao
678719ac5333dfbf8e20607a
Pengfei Wan
claimed_verified
2025-03-17T14:42:53.121Z
662f93942510ef5735d7ad00
https://cdn-avatars.huggingface.co/v1/production/uploads/662f93942510ef5735d7ad00/ZIDIPm63sncIHFTT5b0uR.png
magicwpf
user
magicwpf
678719ac5333dfbf8e20607b
Di Zhang
admin_assigned
2025-01-21T10:08:59.963Z
64bce15bafd1e46c5504ad38
https://cdn-avatars.huggingface.co/v1/production/uploads/64bce15bafd1e46c5504ad38/vkEjiu-mIagKlrXzDH75o.png
Di Zhang
user
di-zhang-fdu
678719ac5333dfbf8e20607c
Xihui Liu
claimed_verified
2025-01-21T10:07:46.546Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
GameFactory: Creating New Games with Generative Interactive Videos
Generative game engines have the potential to revolutionize game development
by autonomously creating new content and reducing manual workload. However,
existing video-based game generation methods fail to address the critical
challenge of scene generalization, limiting their applicability to existing
games with fixed styles and scenes. In this paper, we present GameFactory, a
framework focused on exploring scene generalization in game video generation.
To enable the creation of entirely new and diverse games, we leverage
pre-trained video diffusion models trained on open-domain video data. To bridge
the domain gap between open-domain priors and small-scale game dataset, we
propose a multi-phase training strategy that decouples game style learning from
action control, preserving open-domain generalization while achieving action
controllability. Using Minecraft as our data source, we release GF-Minecraft, a
high-quality and diversity action-annotated video dataset for research.
Furthermore, we extend our framework to enable autoregressive
action-controllable game video generation, allowing the production of
unlimited-length interactive game videos. Experimental results demonstrate that
GameFactory effectively generates open-domain, diverse, and action-controllable
game videos, representing a significant step forward in AI-driven game
generation. Our dataset and project page are publicly available at
https://vvictoryuki.github.io/gamefactory/.
GameFactory leverages pre-trained video diffusion models and a multi-phase training strategy to generate diverse, action-controllable game videos with scene generalization.
video diffusion models scene generalization action-controllability multi-phase training strategy action-annotated video dataset autoregressive model