CheXWorld: Exploring Image World Modeling for Radiograph Representation
  Learning
Humans can develop internal world models that encode common sense knowledge,
telling them how the world works and predicting the consequences of their
actions. This concept has emerged as a promising direction for establishing
general-purpose machine-learning models in recent preliminary works, e.g., for
visual representation learning. In this paper, we present CheXWorld, the first
effort towards a self-supervised world model for radiographic images.
Specifically, our work develops a unified framework that simultaneously models
three aspects of medical knowledge essential for qualified radiologists,
including 1) local anatomical structures describing the fine-grained
characteristics of local tissues (e.g., architectures, shapes, and textures);
2) global anatomical layouts describing the global organization of the human
body (e.g., layouts of organs and skeletons); and 3) domain variations that
encourage CheXWorld to model the transitions across different appearance
domains of radiographs (e.g., varying clarity, contrast, and exposure caused by
collecting radiographs from different hospitals, devices, or patients).
Empirically, we design tailored qualitative and quantitative analyses,
revealing that CheXWorld successfully captures these three dimensions of
medical knowledge. Furthermore, transfer learning experiments across eight
medical image classification and segmentation benchmarks showcase that
CheXWorld significantly outperforms existing SSL methods and large-scale
medical foundation models. Code & pre-trained models are available at
https://github.com/LeapLabTHU/CheXWorld.
6805ab2c40034a5a792a26b2
Yang Yue
claimed_verified
2025-04-22T09:59:59.547Z
63f1d16fbe95ed4c9a9418fe
/avatars/a1bdfa97323693808f2f16ec74698ed3.svg
Yang Yue
user
yueyang2000
6805ab2c40034a5a792a26b3
Yulin Wang
6805ab2c40034a5a792a26b4
Chenxin Tao
6805ab2c40034a5a792a26b5
Pan Liu
6805ab2c40034a5a792a26b6
Shiji Song
6805ab2c40034a5a792a26b7
Gao Huang
CheXWorld: Exploring Image World Modeling for Radiograph Representation
  Learning
Humans can develop internal world models that encode common sense knowledge,
telling them how the world works and predicting the consequences of their
actions. This concept has emerged as a promising direction for establishing
general-purpose machine-learning models in recent preliminary works, e.g., for
visual representation learning. In this paper, we present CheXWorld, the first
effort towards a self-supervised world model for radiographic images.
Specifically, our work develops a unified framework that simultaneously models
three aspects of medical knowledge essential for qualified radiologists,
including 1) local anatomical structures describing the fine-grained
characteristics of local tissues (e.g., architectures, shapes, and textures);
2) global anatomical layouts describing the global organization of the human
body (e.g., layouts of organs and skeletons); and 3) domain variations that
encourage CheXWorld to model the transitions across different appearance
domains of radiographs (e.g., varying clarity, contrast, and exposure caused by
collecting radiographs from different hospitals, devices, or patients).
Empirically, we design tailored qualitative and quantitative analyses,
revealing that CheXWorld successfully captures these three dimensions of
medical knowledge. Furthermore, transfer learning experiments across eight
medical image classification and segmentation benchmarks showcase that
CheXWorld significantly outperforms existing SSL methods and large-scale
medical foundation models. Code & pre-trained models are available at
https://github.com/LeapLabTHU/CheXWorld.
CheXWorld is a self-supervised world model for radiographic images that captures local and global anatomical structures and domain variations, demonstrating superior performance in medical image tasks through transfer learning.
self-supervised world model medical knowledge local anatomical structures global anatomical layouts domain variations transfer learning medical image classification medical image segmentation SSL methods medical foundation models