Self-Improvement in Multimodal Large Language Models: A Survey
Recent advancements in self-improvement for Large Language Models (LLMs) have
efficiently enhanced model capabilities without significantly increasing costs,
particularly in terms of human effort. While this area is still relatively
young, its extension to the multimodal domain holds immense potential for
leveraging diverse data sources and developing more general self-improving
models. This survey is the first to provide a comprehensive overview of
self-improvement in Multimodal LLMs (MLLMs). We provide a structured overview
of the current literature and discuss methods from three perspectives: 1) data
collection, 2) data organization, and 3) model optimization, to facilitate the
further development of self-improvement in MLLMs. We also include commonly used
evaluations and downstream applications. Finally, we conclude by outlining open
challenges and future research directions.
68e31d5073e20ab577841f93
Shijian Deng
68e31d5073e20ab577841f94
Kai Wang
68e31d5073e20ab577841f95
Tianyu Yang
68e31d5073e20ab577841f96
Harsh Singh
68e31d5073e20ab577841f97
Yapeng Tian
Self-Improvement in Multimodal Large Language Models: A Survey
Recent advancements in self-improvement for Large Language Models (LLMs) have
efficiently enhanced model capabilities without significantly increasing costs,
particularly in terms of human effort. While this area is still relatively
young, its extension to the multimodal domain holds immense potential for
leveraging diverse data sources and developing more general self-improving
models. This survey is the first to provide a comprehensive overview of
self-improvement in Multimodal LLMs (MLLMs). We provide a structured overview
of the current literature and discuss methods from three perspectives: 1) data
collection, 2) data organization, and 3) model optimization, to facilitate the
further development of self-improvement in MLLMs. We also include commonly used
evaluations and downstream applications. Finally, we conclude by outlining open
challenges and future research directions.
A survey of self-improvement methods in Multimodal Large Language Models (MLLMs) from data collection, organization, and model optimization perspectives.
Large Language Models Multimodal LLMs self-improvement data collection data organization model optimization evaluations downstream applications open challenges future research directions