Global and Local Entailment Learning for Natural World Imagery
Learning the hierarchical structure of data in vision-language models is a
significant challenge. Previous works have attempted to address this challenge
by employing entailment learning. However, these approaches fail to model the
transitive nature of entailment explicitly, which establishes the relationship
between order and semantics within a representation space. In this work, we
introduce Radial Cross-Modal Embeddings (RCME), a framework that enables the
explicit modeling of transitivity-enforced entailment. Our proposed framework
optimizes for the partial order of concepts within vision-language models. By
leveraging our framework, we develop a hierarchical vision-language foundation
model capable of representing the hierarchy in the Tree of Life. Our
experiments on hierarchical species classification and hierarchical retrieval
tasks demonstrate the enhanced performance of our models compared to the
existing state-of-the-art models. Our code and models are open-sourced at
https://vishu26.github.io/RCME/index.html.
6862a56b3b7fc785c7ca5e07
Srikumar Sastry
claimed_verified
2025-07-03T07:33:06.958Z
64e93050e574e31915800b5e
/avatars/7ff4adb4ffb8b47fe293d0b424baf5d1.svg
Srikumar Sastry
user
Srikumar26
6862a56b3b7fc785c7ca5e08
Aayush Dhakal
6862a56b3b7fc785c7ca5e09
Eric Xing
6862a56b3b7fc785c7ca5e0a
Subash Khanal
6862a56b3b7fc785c7ca5e0b
Nathan Jacobs
Global and Local Entailment Learning for Natural World Imagery
Learning the hierarchical structure of data in vision-language models is a
significant challenge. Previous works have attempted to address this challenge
by employing entailment learning. However, these approaches fail to model the
transitive nature of entailment explicitly, which establishes the relationship
between order and semantics within a representation space. In this work, we
introduce Radial Cross-Modal Embeddings (RCME), a framework that enables the
explicit modeling of transitivity-enforced entailment. Our proposed framework
optimizes for the partial order of concepts within vision-language models. By
leveraging our framework, we develop a hierarchical vision-language foundation
model capable of representing the hierarchy in the Tree of Life. Our
experiments on hierarchical species classification and hierarchical retrieval
tasks demonstrate the enhanced performance of our models compared to the
existing state-of-the-art models. Our code and models are open-sourced at
https://vishu26.github.io/RCME/index.html.
Radial Cross-Modal Embeddings enable explicit modeling of transitive entailment in vision-language models, leading to improved performance in hierarchical species classification and retrieval tasks.
entailment learning Radial Cross-Modal Embeddings (RCME) transitive entailment partial order of concepts hierarchical vision-language foundation model Tree of Life