AnimeGamer: Infinite Anime Life Simulation with Next Game State
  Prediction
Recent advancements in image and video synthesis have opened up new promise
in generative games. One particularly intriguing application is transforming
characters from anime films into interactive, playable entities. This allows
players to immerse themselves in the dynamic anime world as their favorite
characters for life simulation through language instructions. Such games are
defined as infinite game since they eliminate predetermined boundaries and
fixed gameplay rules, where players can interact with the game world through
open-ended language and experience ever-evolving storylines and environments.
Recently, a pioneering approach for infinite anime life simulation employs
large language models (LLMs) to translate multi-turn text dialogues into
language instructions for image generation. However, it neglects historical
visual context, leading to inconsistent gameplay. Furthermore, it only
generates static images, failing to incorporate the dynamics necessary for an
engaging gaming experience. In this work, we propose AnimeGamer, which is built
upon Multimodal Large Language Models (MLLMs) to generate each game state,
including dynamic animation shots that depict character movements and updates
to character states, as illustrated in Figure 1. We introduce novel
action-aware multimodal representations to represent animation shots, which can
be decoded into high-quality video clips using a video diffusion model. By
taking historical animation shot representations as context and predicting
subsequent representations, AnimeGamer can generate games with contextual
consistency and satisfactory dynamics. Extensive evaluations using both
automated metrics and human evaluations demonstrate that AnimeGamer outperforms
existing methods in various aspects of the gaming experience. Codes and
checkpoints are available at https://github.com/TencentARC/AnimeGamer.
67eca389e14049f5ff064ea6
Junhao Cheng
claimed_verified
2025-04-02T08:22:27.228Z
6506b77a773ceaa8d52ecea1
/avatars/0e769a0795063e1491c44760a4a83097.svg
CJH
user
Howe666
67eca389e14049f5ff064ea7
Yuying Ge
admin_assigned
2025-04-03T13:45:57.084Z
6455cc8f654d8bccae50e4d4
/avatars/506a9992e5bf52e06d37cc22e4b307c0.svg
Yuying Ge
user
tttoaster
67eca389e14049f5ff064ea8
Yixiao Ge
admin_assigned
2025-04-03T13:46:03.701Z
640e9762b03f4cd29f58d982
/avatars/81da37d628163fe3e094b247c7c3a3b5.svg
Yixiao Ge
user
yxgeee
67eca389e14049f5ff064ea9
Jing Liao
admin_assigned
2025-04-03T13:46:09.694Z
65e77726767bfc7d109c45bf
/avatars/24e68c86e06055ea1209598ba49ce8b9.svg
Jing Liao
user
CeciliaJL
67eca389e14049f5ff064eaa
Ying Shan
admin_assigned
2025-04-03T13:46:15.314Z
63ca3ddc04c979828310bfcb
/avatars/615e0d8622950b4408b40d550f02a894.svg
Ying Shan
user
yshan2u
AnimeGamer: Infinite Anime Life Simulation with Next Game State
  Prediction
Recent advancements in image and video synthesis have opened up new promise
in generative games. One particularly intriguing application is transforming
characters from anime films into interactive, playable entities. This allows
players to immerse themselves in the dynamic anime world as their favorite
characters for life simulation through language instructions. Such games are
defined as infinite game since they eliminate predetermined boundaries and
fixed gameplay rules, where players can interact with the game world through
open-ended language and experience ever-evolving storylines and environments.
Recently, a pioneering approach for infinite anime life simulation employs
large language models (LLMs) to translate multi-turn text dialogues into
language instructions for image generation. However, it neglects historical
visual context, leading to inconsistent gameplay. Furthermore, it only
generates static images, failing to incorporate the dynamics necessary for an
engaging gaming experience. In this work, we propose AnimeGamer, which is built
upon Multimodal Large Language Models (MLLMs) to generate each game state,
including dynamic animation shots that depict character movements and updates
to character states, as illustrated in Figure 1. We introduce novel
action-aware multimodal representations to represent animation shots, which can
be decoded into high-quality video clips using a video diffusion model. By
taking historical animation shot representations as context and predicting
subsequent representations, AnimeGamer can generate games with contextual
consistency and satisfactory dynamics. Extensive evaluations using both
automated metrics and human evaluations demonstrate that AnimeGamer outperforms
existing methods in various aspects of the gaming experience. Codes and
checkpoints are available at https://github.com/TencentARC/AnimeGamer.
AnimeGamer uses Multimodal Large Language Models (MLLMs) and a video diffusion model to generate contextually consistent, dynamic, and high-quality video clips for an infinite game simulation.
Multimodal Large Language Models MLLMs video diffusion model action-aware multimodal representations dynamic animation shots historical animation shot representations contextually consistent