Understanding Visual Feature Reliance through the Lens of Complexity
Recent studies suggest that deep learning models inductive bias towards
favoring simpler features may be one of the sources of shortcut learning. Yet,
there has been limited focus on understanding the complexity of the myriad
features that models learn. In this work, we introduce a new metric for
quantifying feature complexity, based on V-information and
capturing whether a feature requires complex computational transformations to
be extracted. Using this V-information metric, we analyze the
complexities of 10,000 features, represented as directions in the penultimate
layer, that were extracted from a standard ImageNet-trained vision model. Our
study addresses four key questions: First, we ask what features look like as a
function of complexity and find a spectrum of simple to complex features
present within the model. Second, we ask when features are learned during
training. We find that simpler features dominate early in training, and more
complex features emerge gradually. Third, we investigate where within the
network simple and complex features flow, and find that simpler features tend
to bypass the visual hierarchy via residual connections. Fourth, we explore the
connection between features complexity and their importance in driving the
networks decision. We find that complex features tend to be less important.
Surprisingly, important features become accessible at earlier layers during
training, like a sedimentation process, allowing the model to build upon these
foundational elements.
668db3b3e990292e5fbb6553
Thomas Fel
admin_assigned
2024-07-10T11:56:40.525Z
620dbbf8611163230f7d0fae
https://cdn-avatars.huggingface.co/v1/production/uploads/620dbbf8611163230f7d0fae/xDqa6GMiSBhsb9G_9jdne.jpeg
Thomas FEL
user
ThomasFEL
668db3b3e990292e5fbb6554
Louis Bethune
admin_assigned
2024-07-10T11:56:47.617Z
662a8d7faa7781f8ea521f16
/avatars/c05f9a7885b89c8818d52b98b52ecc7b.svg
Louis Bethune
user
AlgueRythme
668db3b3e990292e5fbb6555
Andrew Kyle Lampinen
668db3b3e990292e5fbb6556
Thomas Serre
admin_assigned
2024-07-10T11:57:00.057Z
64c134cb7fd776c64945f066
/avatars/dea3c78673dd0d430e267acd310fb79c.svg
Thomas Serre
user
tserre
668db3b3e990292e5fbb6557
Katherine Hermann
Understanding Visual Feature Reliance through the Lens of Complexity
Recent studies suggest that deep learning models inductive bias towards
favoring simpler features may be one of the sources of shortcut learning. Yet,
there has been limited focus on understanding the complexity of the myriad
features that models learn. In this work, we introduce a new metric for
quantifying feature complexity, based on V-information and
capturing whether a feature requires complex computational transformations to
be extracted. Using this V-information metric, we analyze the
complexities of 10,000 features, represented as directions in the penultimate
layer, that were extracted from a standard ImageNet-trained vision model. Our
study addresses four key questions: First, we ask what features look like as a
function of complexity and find a spectrum of simple to complex features
present within the model. Second, we ask when features are learned during
training. We find that simpler features dominate early in training, and more
complex features emerge gradually. Third, we investigate where within the
network simple and complex features flow, and find that simpler features tend
to bypass the visual hierarchy via residual connections. Fourth, we explore the
connection between features complexity and their importance in driving the
networks decision. We find that complex features tend to be less important.
Surprisingly, important features become accessible at earlier layers during
training, like a sedimentation process, allowing the model to build upon these
foundational elements.
A new $\mathscr{V}$-information metric is introduced to quantify feature complexity in deep learning models, revealing a spectrum of features, their acquisition timeline, flow within the network, and their importance in decision-making.
$\mathscr{V}$-information feature complexity penultimate layer residual connections feature importance visual hierarchy