PaLI-X: On Scaling up a Multilingual Vision and Language Model
We present the training recipe and results of scaling up PaLI-X, a
multilingual vision and language model, both in terms of size of the components
and the breadth of its training task mixture. Our model achieves new levels of
performance on a wide-range of varied and complex tasks, including multiple
image-based captioning and question-answering tasks, image-based document
understanding and few-shot (in-context) learning, as well as object detection,
video question answering, and video captioning. PaLI-X advances the
state-of-the-art on most vision-and-language benchmarks considered (25+ of
them). Finally, we observe emerging capabilities, such as complex counting and
multilingual object detection, tasks that are not explicitly in the training
mix.
6476c48d9b260a1739bb981b
Xi Chen
6476c48d9b260a1739bb981c
Josip Djolonga
6476c48d9b260a1739bb981d
Piotr Padlewski
claimed_verified
2023-05-31T12:01:15.940Z
64770ddafe51edaf1fc0322d
https://cdn-avatars.huggingface.co/v1/production/uploads/64770ddafe51edaf1fc0322d/jxN1_ALBmz7hn-IG8q75P.jpeg
Piotr Padlewski
user
prazek
6476c48d9b260a1739bb981e
Basil Mustafa
admin_assigned
2023-05-31T08:03:43.926Z
63cfcad6e23b90128c66685c
https://cdn-avatars.huggingface.co/v1/production/uploads/1674562206311-noauth.png
Basil Mustafa
user
BasilMustafa
6476c48d9b260a1739bb981f
Soravit Changpinyo
6476c48d9b260a1739bb9820
Jialin Wu
6476c48d9b260a1739bb9821
Carlos Riquelme Ruiz
6476c48d9b260a1739bb9822
Sebastian Goodman
6476c48d9b260a1739bb9823
Xiao Wang
6476c48d9b260a1739bb9824
Yi Tay
6476c48d9b260a1739bb9825
Siamak Shakeri
6476c48d9b260a1739bb9826
Mostafa Dehghani
6476c48d9b260a1739bb9827
Daniel Salz
claimed_verified
2025-09-26T12:30:52.752Z
68d5c05805ddd80041f80776
/avatars/53a304076359f11cc92de22a2cfbec06.svg
Daniel Salz
user
dasalz
6476c48d9b260a1739bb9828
Mario Lucic
admin_assigned
2023-05-31T08:05:49.667Z
643e8cc492159b909ffe8dd2
/avatars/fc0b506916912e60971376997b3941ba.svg
Mario Lucic
user
lucic
6476c48d9b260a1739bb9829
Michael Tschannen
2023-05-31T08:06:07.746Z
6476c48d9b260a1739bb982a
Arsha Nagrani
6476c48d9b260a1739bb982b
Hexiang Hu
admin_assigned
2023-05-31T08:06:47.365Z
643441ccd55dea2d0ec2c309
/avatars/82e99d445e2b513ad7270fa852adbcbb.svg
Hexiang Hu
user
hexianghu
6476c48d9b260a1739bb982c
Mandar Joshi
admin_assigned
2023-05-31T08:07:07.413Z
640f5e6cc025ddf618988565
/avatars/d3983ed6f7b909d6b5c6b7f68408fefd.svg
Mandar Joshi
user
mandarjoshi
6476c48d9b260a1739bb982d
Bo Pang
6476c48d9b260a1739bb982e
Ceslee Montgomery
6476c48d9b260a1739bb982f
Paulina Pietrzyk
6476c48d9b260a1739bb9830
Marvin Ritter
6476c48d9b260a1739bb9831
AJ Piergiovanni
6476c48d9b260a1739bb9832
Matthias Minderer
6476c48d9b260a1739bb9833
Filip Pavetic
6476c48d9b260a1739bb9834
Austin Waters
6476c48d9b260a1739bb9835
Gang Li
6476c48d9b260a1739bb9836
Ibrahim Alabdulmohsin
admin_assigned
2023-05-31T08:08:24.966Z
630545da20668afe24860235
/avatars/5d82be2e7412bff1af15cc5eafa60b7d.svg
Ibrahim Alabdulmohsin
user
ibomohsin
6476c48d9b260a1739bb9837
Lucas Beyer
admin_assigned
2023-05-31T08:08:49.062Z
642d334ff65714b4585f2de4
https://cdn-avatars.huggingface.co/v1/production/uploads/642d334ff65714b4585f2de4/gxBynq5KyoUP0VlAQD3-w.jpeg
Lucas Beyer
user
giffmana
6476c48d9b260a1739bb9838
Julien Amelot
6476c48d9b260a1739bb9839
Kenton Lee
6476c48d9b260a1739bb983a
Andreas Peter Steiner
6476c48d9b260a1739bb983b
Yang Li
6476c48d9b260a1739bb983c
Daniel Keysers
claimed_verified
2024-07-11T10:05:34.589Z
662fc73ccee11621a1067db1
/avatars/4fbfbc0621f631d8e95a2a642fa0cd27.svg
Daniel Keysers
user
dkeysers
6476c48d9b260a1739bb983d
Anurag Arnab
admin_assigned
2023-05-31T08:10:14.969Z
63d7f0466ac3104e50c5311e
/avatars/2421bfb7747664d1acefbac1c00c3475.svg
Anurag Arnab
user
anurag-arnab
6476c48d9b260a1739bb983e
Yuanzhong Xu
6476c48d9b260a1739bb983f
Keran Rong
claimed_verified
2025-02-13T08:28:05.635Z
648a9083abcf427a9a498679
/avatars/e32d413ce0a48d83f95e29c11a8a8ae8.svg
Keran 
user
Keeera
6476c48d9b260a1739bb9840
Alexander Kolesnikov
6476c48d9b260a1739bb9841
Mojtaba Seyedhosseini
6476c48d9b260a1739bb9842
Anelia Angelova
6476c48d9b260a1739bb9843
Xiaohua Zhai
6476c48d9b260a1739bb9844
Neil Houlsby
6476c48d9b260a1739bb9845
Radu Soricut
PaLI-X: On Scaling up a Multilingual Vision and Language Model
We present the training recipe and results of scaling up PaLI-X, a
multilingual vision and language model, both in terms of size of the components
and the breadth of its training task mixture. Our model achieves new levels of
performance on a wide-range of varied and complex tasks, including multiple
image-based captioning and question-answering tasks, image-based document
understanding and few-shot (in-context) learning, as well as object detection,
video question answering, and video captioning. PaLI-X advances the
state-of-the-art on most vision-and-language benchmarks considered (25+ of
them). Finally, we observe emerging capabilities, such as complex counting and
multilingual object detection, tasks that are not explicitly in the training
mix.
PaLI-X, a multilingual vision and language model, achieves state-of-the-art performance on various tasks by scaling its components and task diversity.
multilingual vision and language model image-based captioning image-based question-answering image-based document understanding few-shot learning object detection video question answering video captioning vision-and-language benchmarks complex counting multilingual object detection