Platypus: A Generalized Specialist Model for Reading Text in Various
  Forms
Reading text from images (either natural scenes or documents) has been a
long-standing research topic for decades, due to the high technical challenge
and wide application range. Previously, individual specialist models are
developed to tackle the sub-tasks of text reading (e.g., scene text
recognition, handwritten text recognition and mathematical expression
recognition). However, such specialist models usually cannot effectively
generalize across different sub-tasks. Recently, generalist models (such as
GPT-4V), trained on tremendous data in a unified way, have shown enormous
potential in reading text in various scenarios, but with the drawbacks of
limited accuracy and low efficiency. In this work, we propose Platypus, a
generalized specialist model for text reading. Specifically, Platypus combines
the best of both worlds: being able to recognize text of various forms with a
single unified architecture, while achieving excellent accuracy and high
efficiency. To better exploit the advantage of Platypus, we also construct a
text reading dataset (called Worms), the images of which are curated from
previous datasets and partially re-labeled. Experiments on standard benchmarks
demonstrate the effectiveness and superiority of the proposed Platypus model.
Model and data will be made publicly available at
https://github.com/AlibabaResearch/AdvancedLiterateMachinery/tree/main/OCR/Platypus.
66ce83b2da5d6bbe4d002b25
Peng Wang
66ce83b2da5d6bbe4d002b26
Zhaohai Li
66ce83b2da5d6bbe4d002b27
Jun Tang
66ce83b2da5d6bbe4d002b28
Humen Zhong
66ce83b2da5d6bbe4d002b29
Fei Huang
admin_assigned
2024-08-28T09:34:52.501Z
635b8b6a37c6a2c12e2cce00
/avatars/229fb72180529141515d1df797b33709.svg
Fei Huang
user
hzhwcmhf
66ce83b2da5d6bbe4d002b2a
Zhibo Yang
admin_assigned
2024-08-28T09:34:13.970Z
66051de91b8220d71becb448
/avatars/f2b507f5821f8baa90b60f62ae4de389.svg
杨智博
user
zhiboyang
66ce83b2da5d6bbe4d002b2b
Cong Yao
admin_assigned
2024-08-28T09:34:03.965Z
6638f898c168eec02354abeb
/avatars/0a0b7a1b056463fb4ccf6bfe6456a7d1.svg
Cong Yao
user
bridgetop3young
Platypus: A Generalized Specialist Model for Reading Text in Various
  Forms
Reading text from images (either natural scenes or documents) has been a
long-standing research topic for decades, due to the high technical challenge
and wide application range. Previously, individual specialist models are
developed to tackle the sub-tasks of text reading (e.g., scene text
recognition, handwritten text recognition and mathematical expression
recognition). However, such specialist models usually cannot effectively
generalize across different sub-tasks. Recently, generalist models (such as
GPT-4V), trained on tremendous data in a unified way, have shown enormous
potential in reading text in various scenarios, but with the drawbacks of
limited accuracy and low efficiency. In this work, we propose Platypus, a
generalized specialist model for text reading. Specifically, Platypus combines
the best of both worlds: being able to recognize text of various forms with a
single unified architecture, while achieving excellent accuracy and high
efficiency. To better exploit the advantage of Platypus, we also construct a
text reading dataset (called Worms), the images of which are curated from
previous datasets and partially re-labeled. Experiments on standard benchmarks
demonstrate the effectiveness and superiority of the proposed Platypus model.
Model and data will be made publicly available at
https://github.com/AlibabaResearch/AdvancedLiterateMachinery/tree/main/OCR/Platypus.
scene text recognition handwritten text recognition mathematical expression recognition generalist models GPT-4V generalized specialist model dual unified architecture Platypus Worms dataset standard benchmarks