CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster
  Pre-training on Web-scale Image-Text Data
Contrastive learning has emerged as a transformative method for learning
effective visual representations through the alignment of image and text
embeddings. However, pairwise similarity computation in contrastive loss
between image and text pairs poses computational challenges. This paper
presents a novel weakly supervised pre-training of vision models on web-scale
image-text data. The proposed method reframes pre-training on image-text data
as a classification task. Consequently, it eliminates the need for pairwise
similarity computations in contrastive loss, achieving a remarkable 2.7times
acceleration in training speed compared to contrastive learning on web-scale
data. Through extensive experiments spanning diverse vision tasks, including
detection and segmentation, we demonstrate that the proposed method maintains
high representation quality. Our source code along with pre-trained model
weights and training recipes is available at
https://github.com/apple/corenet.
662a9a5b565dd1ec4801530a
Sachin Mehta
admin_assigned
2024-04-26T12:06:04.810Z
64f792652003abb61bf5c1b8
/avatars/9721810de3519ece50b059c154baa9ca.svg
Sachin Mehta
user
sacmehta
662a9a5b565dd1ec4801530b
Maxwell Horton
admin_assigned
2024-04-26T12:06:12.519Z
647a5afc732a0cd866eb512f
/avatars/fbd0ba2a7ba70414ac903e583ce62933.svg
Maxwell Horton
user
mchorton
662a9a5b565dd1ec4801530c
Fartash Faghri
admin_assigned
2024-04-26T12:06:18.620Z
65f8945a3ba58880f9794a11
/avatars/fde0ea1c20f89b3a83dbefb24e728d61.svg
Fartash Faghri
user
fartashf
662a9a5b565dd1ec4801530d
Mohammad Hossein Sekhavat
admin_assigned
2024-04-26T12:06:25.187Z
65ca91986938a6a81fd80d02
/avatars/efac7945ac1df4e8949e77465ef0dcbd.svg
Mohammad Hossein Sekhavat
user
msekhavat
662a9a5b565dd1ec4801530e
Mahyar Najibi
admin_assigned
2024-04-26T12:06:47.100Z
661d992d69a65c0fb0717013
/avatars/9d1f20baff742de90a50232db54d2407.svg
Mahyar Najibi
user
mahyar-najibi
662a9a5b565dd1ec4801530f
Mehrdad Farajtabar
admin_assigned
2024-04-26T12:06:55.648Z
62d188a6d50433c35eb14020
/avatars/fc3d149a27bc8396010a2a02148e8ca0.svg
Mehrdad Farajtabar
user
mfarajtabar
662a9a5b565dd1ec48015310
Oncel Tuzel
662a9a5b565dd1ec48015311
Mohammad Rastegari
admin_assigned
2024-04-26T12:07:05.928Z
65d4c7734c2163d05eab4110
/avatars/b116a9aca7b0a2a4836a29cb4f069d42.svg
Mohammad Rastegari
user
mrastegari
CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster
  Pre-training on Web-scale Image-Text Data
Contrastive learning has emerged as a transformative method for learning
effective visual representations through the alignment of image and text
embeddings. However, pairwise similarity computation in contrastive loss
between image and text pairs poses computational challenges. This paper
presents a novel weakly supervised pre-training of vision models on web-scale
image-text data. The proposed method reframes pre-training on image-text data
as a classification task. Consequently, it eliminates the need for pairwise
similarity computations in contrastive loss, achieving a remarkable 2.7times
acceleration in training speed compared to contrastive learning on web-scale
data. Through extensive experiments spanning diverse vision tasks, including
detection and segmentation, we demonstrate that the proposed method maintains
high representation quality. Our source code along with pre-trained model
weights and training recipes is available at
https://github.com/apple/corenet.
A novel weakly supervised pre-training method for vision models on web-scale data reframes the task as classification to avoid pairwise similarity computations, significantly speeding up training while maintaining high representation quality.
contrastive learning image-text embeddings contrastive loss weakly supervised pre-training classification task