Automatic Creative Selection with Cross-Modal Matching
Application developers advertise their Apps by creating product pages with
App images, and bidding on search terms. It is then crucial for App images to
be highly relevant with the search terms. Solutions to this problem require an
image-text matching model to predict the quality of the match between the
chosen image and the search terms. In this work, we present a novel approach to
matching an App image to search terms based on fine-tuning a pre-trained LXMERT
model. We show that compared to the CLIP model and a baseline using a
Transformer model for search terms, and a ResNet model for images, we
significantly improve the matching accuracy. We evaluate our approach using two
sets of labels: advertiser associated (image, search term) pairs for a given
application, and human ratings for the relevance between (image, search term)
pairs. Our approach achieves 0.96 AUC score for advertiser associated ground
truth, outperforming the transformer+ResNet baseline and the fine-tuned CLIP
model by 8% and 14%. For human labeled ground truth, our approach achieves 0.95
AUC score, outperforming the transformer+ResNet baseline and the fine-tuned
CLIP model by 16% and 17%.
66332a4b5ddb7702ad419478
Alex Kim
66332a4b5ddb7702ad419479
Jia Huang
extracted_pending
2024-05-02T05:53:15.711Z
64b86a14f1f8e6ea584dd4dc
/avatars/dc365aff617f5dfd75f44d5b013065bc.svg
Jia Huang
user
jiahuang-5
66332a4b5ddb7702ad41947a
Rob Monarch
66332a4b5ddb7702ad41947b
Jerry Kwac
66332a4b5ddb7702ad41947c
Anikesh Kamath
66332a4b5ddb7702ad41947d
Parmeshwar Khurd
66332a4b5ddb7702ad41947e
Kailash Thiyagarajan
admin_assigned
2024-05-02T07:42:59.039Z
661b2261a12a10bd0d997f77
/avatars/f924eafa3e2bba1d6e07782aba04f762.svg
Kailash Thiyagarajan
user
tkailash
66332a4b5ddb7702ad41947f
Goodman Gu
admin_assigned
2024-05-02T07:43:05.334Z
6472ef5fd9e9598973c6c558
/avatars/8a184f6c3754813730a7cb7e70a75df7.svg
Goodman Gu
user
vivaldi2007
Automatic Creative Selection with Cross-Modal Matching
Application developers advertise their Apps by creating product pages with
App images, and bidding on search terms. It is then crucial for App images to
be highly relevant with the search terms. Solutions to this problem require an
image-text matching model to predict the quality of the match between the
chosen image and the search terms. In this work, we present a novel approach to
matching an App image to search terms based on fine-tuning a pre-trained LXMERT
model. We show that compared to the CLIP model and a baseline using a
Transformer model for search terms, and a ResNet model for images, we
significantly improve the matching accuracy. We evaluate our approach using two
sets of labels: advertiser associated (image, search term) pairs for a given
application, and human ratings for the relevance between (image, search term)
pairs. Our approach achieves 0.96 AUC score for advertiser associated ground
truth, outperforming the transformer+ResNet baseline and the fine-tuned CLIP
model by 8% and 14%. For human labeled ground truth, our approach achieves 0.95
AUC score, outperforming the transformer+ResNet baseline and the fine-tuned
CLIP model by 16% and 17%.
A novel image-text matching model fine-tuned on LXMERT improves the accuracy of matching app images to search terms compared to CLIP and a transformer-ResNet baseline.
LXMERT CLIP Transformer ResNet image-text matching AUC score