VisualLens: Personalization through Visual History
We hypothesize that a user's visual history with images reflecting their
daily life, offers valuable insights into their interests and preferences, and
can be leveraged for personalization. Among the many challenges to achieve this
goal, the foremost is the diversity and noises in the visual history,
containing images not necessarily related to a recommendation task, not
necessarily reflecting the user's interest, or even not necessarily
preference-relevant. Existing recommendation systems either rely on
task-specific user interaction logs, such as online shopping history for
shopping recommendations, or focus on text signals. We propose a novel
approach, VisualLens, that extracts, filters, and refines image
representations, and leverages these signals for personalization. We created
two new benchmarks with task-agnostic visual histories, and show that our
method improves over state-of-the-art recommendations by 5-10% on Hit@3, and
improves over GPT-4o by 2-5%. Our approach paves the way for personalized
recommendations in scenarios where traditional methods fail.
67455f372ec5738026c814d5
Wang Bill Zhu
claimed_verified
2024-11-26T21:20:55.505Z
6389568eec1f539adc0e0337
/avatars/6ee5c9bea9989e4618ae8d334bd889a9.svg
Wang Bill Zhu
user
Bill1235813
67455f372ec5738026c814d6
Deqing Fu
claimed_verified
2024-11-26T21:20:53.899Z
63c8454e46421a2efe82709d
https://cdn-avatars.huggingface.co/v1/production/uploads/63c8454e46421a2efe82709d/3BcSk4KOwAgWHEPVtsAV3.png
Deqing Fu
user
deqing
67455f372ec5738026c814d7
Kai Sun
67455f372ec5738026c814d8
Yi Lu
67455f372ec5738026c814d9
Zhaojiang Lin
admin_assigned
2024-11-26T22:48:27.497Z
64b6d095dbbd1f2cdb62a946
/avatars/b5f97735045d05ba5d30ee1fbf3f9bae.svg
Zhaojiang Lin
user
zlinao
67455f372ec5738026c814da
Seungwhan Moon
admin_assigned
2024-11-26T22:47:40.512Z
64c9ae24e761f47061448bfb
/avatars/8835bcf769fd738e8c66fa7a52ddc421.svg
Seungwhan Moon
user
shanemoon27
67455f372ec5738026c814db
Kanika Narang
admin_assigned
2024-11-26T22:47:47.839Z
66cd426500cddde87a6404f0
/avatars/06ab2ee091e3912688c5d6ed491d2640.svg
Kanika Narang
user
Kanika91
67455f372ec5738026c814dc
Mustafa Canim
admin_assigned
2024-11-26T22:47:53.221Z
6657cd704018967357c5e135
/avatars/cfa39b7cf8c00da1a6594c866e3d90df.svg
Mustafa Canim
user
mustafacanim
67455f372ec5738026c814dd
Yue Liu
claimed_verified
2024-11-27T22:20:51.000Z
66671b27ad9cd05ea74202a5
/avatars/a6fa26c173f3e5b0748e5fb007c3105e.svg
Yue Liu
user
liu-yue
67455f372ec5738026c814de
Anuj Kumar
67455f372ec5738026c814df
Xin Luna Dong
VisualLens: Personalization through Visual History
We hypothesize that a user's visual history with images reflecting their
daily life, offers valuable insights into their interests and preferences, and
can be leveraged for personalization. Among the many challenges to achieve this
goal, the foremost is the diversity and noises in the visual history,
containing images not necessarily related to a recommendation task, not
necessarily reflecting the user's interest, or even not necessarily
preference-relevant. Existing recommendation systems either rely on
task-specific user interaction logs, such as online shopping history for
shopping recommendations, or focus on text signals. We propose a novel
approach, VisualLens, that extracts, filters, and refines image
representations, and leverages these signals for personalization. We created
two new benchmarks with task-agnostic visual histories, and show that our
method improves over state-of-the-art recommendations by 5-10% on Hit@3, and
improves over GPT-4o by 2-5%. Our approach paves the way for personalized
recommendations in scenarios where traditional methods fail.
VisualLens extracts, filters, and refines image representations from users' visual histories to enhance personalization, outperforming existing methods and GPT-4o in recommendation tasks.
image representations visual histories personalization Hit@3 GPT-4o