BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and
  Vision-Language Models Derived from Scientific Literature
The development of vision-language models (VLMs) is driven by large-scale and
diverse multimodal datasets. However, progress toward generalist biomedical
VLMs is limited by the lack of annotated, publicly accessible datasets across
biology and medicine. Existing efforts are restricted to narrow domains,
missing the full diversity of biomedical knowledge encoded in scientific
literature. To address this gap, we introduce BIOMEDICA, a scalable,
open-source framework to extract, annotate, and serialize the entirety of the
PubMed Central Open Access subset into an easy-to-use, publicly accessible
dataset.Our framework produces a comprehensive archive with over 24 million
unique image-text pairs from over 6 million articles. Metadata and
expert-guided annotations are also provided. We demonstrate the utility and
accessibility of our resource by releasing BMCA-CLIP, a suite of CLIP-style
models continuously pre-trained on the BIOMEDICA dataset via streaming,
eliminating the need to download 27 TB of data locally.On average, our models
achieve state-of-the-art performance across 40 tasks - spanning pathology,
radiology, ophthalmology, dermatology, surgery, molecular biology,
parasitology, and cell biology - excelling in zero-shot classification with a
6.56% average improvement (as high as 29.8% and 17.5% in dermatology and
ophthalmology, respectively), and stronger image-text retrieval, all while
using 10x less compute. To foster reproducibility and collaboration, we release
our codebase and dataset for the broader research community.
678603c0c054524aa22be7e1
Alejandro Lozano
extracted_pending
2025-01-14T06:27:14.763Z
643c3fb526f177a3e4181e05
/avatars/4f08ca15c5fbd2f8bb3d5c24f3fcc7b8.svg
Alejandro Lozano
user
lozanoe
678603c0c054524aa22be7e2
Min Woo Sun
extracted_confirmed
2025-01-14T06:45:53.756Z
65ac61120844d9e0d67a9f89
https://cdn-avatars.huggingface.co/v1/production/uploads/65ac61120844d9e0d67a9f89/OzA_yxFoeuyhmc3xdYYdP.jpeg
Min Woo Sun
user
minwoosun
678603c0c054524aa22be7e3
James Burgess
claimed_verified
2025-01-15T08:49:11.461Z
650871aeb44445e9b3625c7b
https://cdn-avatars.huggingface.co/v1/production/uploads/650871aeb44445e9b3625c7b/mtx3EnkuNF4z29IosnhaQ.png
James Burgess
user
jmhb
678603c0c054524aa22be7e4
Liangyu Chen
claimed_verified
2025-01-15T08:49:09.051Z
62b67da0f56de4396ca9e44b
https://cdn-avatars.huggingface.co/v1/production/uploads/1658586059273-62b67da0f56de4396ca9e44b.jpeg
Liangyu Chen
user
liangyuch
678603c0c054524aa22be7e5
Jeffrey J Nirschl
extracted_pending
2025-01-14T06:27:14.763Z
6500e67e13f1546526bd373a
/avatars/80ef1ce58e93b773dcbcc2d5b53175c2.svg
Jeff Nirschl
user
jnirschl
678603c0c054524aa22be7e6
Jeffrey Gu
678603c0c054524aa22be7e7
Ivan Lopez
claimed_verified
2025-01-15T08:49:06.626Z
653fe33e73325383c2c9f9ba
/avatars/9f8863fbbc613915c70e620b7a147e4d.svg
Ivan Lopez
user
ivlopez
678603c0c054524aa22be7e8
Josiah Aklilu
claimed_verified
2025-01-16T21:55:50.124Z
66844ab799dbd7c30a19a0fe
/avatars/5a5f8fa530e127105812bd1d0e515db8.svg
Josiah Aklilu
user
josaklil-ai
678603c0c054524aa22be7e9
Austin Wolfgang Katzer
extracted_pending
2025-01-15T00:04:26.115Z
6786fb7ade13ae3961c03d4d
/avatars/156658b3c24283a564472001dec63e5a.svg
Austin Katzer
user
Awkatzer
678603c0c054524aa22be7ea
Collin Chiu
678603c0c054524aa22be7eb
Anita Rau
claimed_verified
2025-02-18T09:35:17.795Z
6733c7f1d3521b36243b3c91
/avatars/a070c5296fbce770822b1897157ddbe3.svg
Anita
user
anitar03
678603c0c054524aa22be7ec
Xiaohan Wang
claimed_verified
2025-01-16T08:31:48.670Z
65703fab7f50602340d23704
/avatars/324c45f5fba9cd8c38a89b30427c06b4.svg
Xiaohan Wang
user
nicholswang
678603c0c054524aa22be7ed
Yuhui Zhang
extracted_confirmed
2025-01-14T06:28:22.730Z
62da55164398e21bf7f0e292
https://cdn-avatars.huggingface.co/v1/production/uploads/62da55164398e21bf7f0e292/xjKkG8IA2IZZqCdjApSh3.jpeg
Yuhui Zhang
user
yuhuizhang
678603c0c054524aa22be7ee
Alfred Seunghoon Song
678603c0c054524aa22be7ef
Robert Tibshirani
678603c0c054524aa22be7f0
Serena Yeung-Levy
extracted_pending
2025-01-14T06:27:14.763Z
677c8b2e92550a07fcad0f50
/avatars/2be26e8f25e98cfe5b1d227ee0409cd0.svg
Serena Yeung-Levy
user
yeunglevy
BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and
  Vision-Language Models Derived from Scientific Literature
The development of vision-language models (VLMs) is driven by large-scale and
diverse multimodal datasets. However, progress toward generalist biomedical
VLMs is limited by the lack of annotated, publicly accessible datasets across
biology and medicine. Existing efforts are restricted to narrow domains,
missing the full diversity of biomedical knowledge encoded in scientific
literature. To address this gap, we introduce BIOMEDICA, a scalable,
open-source framework to extract, annotate, and serialize the entirety of the
PubMed Central Open Access subset into an easy-to-use, publicly accessible
dataset.Our framework produces a comprehensive archive with over 24 million
unique image-text pairs from over 6 million articles. Metadata and
expert-guided annotations are also provided. We demonstrate the utility and
accessibility of our resource by releasing BMCA-CLIP, a suite of CLIP-style
models continuously pre-trained on the BIOMEDICA dataset via streaming,
eliminating the need to download 27 TB of data locally.On average, our models
achieve state-of-the-art performance across 40 tasks - spanning pathology,
radiology, ophthalmology, dermatology, surgery, molecular biology,
parasitology, and cell biology - excelling in zero-shot classification with a
6.56% average improvement (as high as 29.8% and 17.5% in dermatology and
ophthalmology, respectively), and stronger image-text retrieval, all while
using 10x less compute. To foster reproducibility and collaboration, we release
our codebase and dataset for the broader research community.
BIOMEDICA is a scalable framework that creates a large, publicly accessible dataset of image-text pairs from PubMed Central, enabling state-of-the-art performance across various biomedical tasks with efficient CLIP-style models.
vision-language models VLMs multimodal datasets BIOMEDICA PubMed Central Open Access image-text pairs Metadata expert-guided annotations BMCA-CLIP CLIP-style models streaming zero-shot classification image-text retrieval