Processing and acquisition traces in visual encoders: What does CLIP
  know about your camera?
Prior work has analyzed the robustness of visual encoders to image
transformations and corruptions, particularly in cases where such alterations
are not seen during training. When this occurs, they introduce a form of
distribution shift at test time, often leading to performance degradation. The
primary focus has been on severe corruptions that, when applied aggressively,
distort useful signals necessary for accurate semantic predictions.
  We take a different perspective by analyzing parameters of the image
acquisition process and transformations that may be subtle or even
imperceptible to the human eye. We find that such parameters are systematically
encoded in the learned visual representations and can be easily recovered. More
strikingly, their presence can have a profound impact, either positively or
negatively, on semantic predictions. This effect depends on whether there is a
strong correlation or anti-correlation between semantic labels and these
acquisition-based or processing-based labels. Our code and data are available
at: https://github.com/ryan-caesar-ramos/visual-encoder-traces
689eee00a4caabb4320e5e5b
Ryan Ramos
claimed_verified
2025-08-18T06:56:55.529Z
62cf7c6526c94b143177de34
https://cdn-avatars.huggingface.co/v1/production/uploads/62cf7c6526c94b143177de34/vvKlleIXxsllVWDi1ySdx.jpeg
Ryan Ramos
user
ryanramos
689eee00a4caabb4320e5e5c
Vladan Stojnić
claimed_verified
2025-08-18T06:56:57.853Z
66a3ae59f33ff23e1c027ccd
https://cdn-avatars.huggingface.co/v1/production/uploads/66a3ae59f33ff23e1c027ccd/tZzpESNPnmhty62xhHszF.jpeg
Vladan Stojnic
user
stojnvla
689eee00a4caabb4320e5e5d
Giorgos Kordopatis-Zilos
689eee00a4caabb4320e5e5e
Yuta Nakashima
689eee00a4caabb4320e5e5f
Giorgos Tolias
689eee00a4caabb4320e5e60
Noa Garcia
Processing and acquisition traces in visual encoders: What does CLIP
  know about your camera?
Prior work has analyzed the robustness of visual encoders to image
transformations and corruptions, particularly in cases where such alterations
are not seen during training. When this occurs, they introduce a form of
distribution shift at test time, often leading to performance degradation. The
primary focus has been on severe corruptions that, when applied aggressively,
distort useful signals necessary for accurate semantic predictions.
  We take a different perspective by analyzing parameters of the image
acquisition process and transformations that may be subtle or even
imperceptible to the human eye. We find that such parameters are systematically
encoded in the learned visual representations and can be easily recovered. More
strikingly, their presence can have a profound impact, either positively or
negatively, on semantic predictions. This effect depends on whether there is a
strong correlation or anti-correlation between semantic labels and these
acquisition-based or processing-based labels. Our code and data are available
at: https://github.com/ryan-caesar-ramos/visual-encoder-traces
Visual encoders encode subtle image acquisition parameters that can significantly impact semantic predictions based on their correlation with semantic labels.
visual encoders image acquisition process transformations learned visual representations semantic predictions acquisition-based labels processing-based labels