Frequency Dynamic Convolution for Dense Image Prediction
While Dynamic Convolution (DY-Conv) has shown promising performance by
enabling adaptive weight selection through multiple parallel weights combined
with an attention mechanism, the frequency response of these weights tends to
exhibit high similarity, resulting in high parameter costs but limited
adaptability. In this work, we introduce Frequency Dynamic Convolution
(FDConv), a novel approach that mitigates these limitations by learning a fixed
parameter budget in the Fourier domain. FDConv divides this budget into
frequency-based groups with disjoint Fourier indices, enabling the construction
of frequency-diverse weights without increasing the parameter cost. To further
enhance adaptability, we propose Kernel Spatial Modulation (KSM) and Frequency
Band Modulation (FBM). KSM dynamically adjusts the frequency response of each
filter at the spatial level, while FBM decomposes weights into distinct
frequency bands in the frequency domain and modulates them dynamically based on
local content. Extensive experiments on object detection, segmentation, and
classification validate the effectiveness of FDConv. We demonstrate that when
applied to ResNet-50, FDConv achieves superior performance with a modest
increase of +3.6M parameters, outperforming previous methods that require
substantial increases in parameter budgets (e.g., CondConv +90M, KW +76.5M).
Moreover, FDConv seamlessly integrates into a variety of architectures,
including ConvNeXt, Swin-Transformer, offering a flexible and efficient
solution for modern vision tasks. The code is made publicly available at
https://github.com/Linwei-Chen/FDConv.
67e2a43d5116df47da357eec
Linwei Chen
claimed_verified
2025-03-25T14:36:22.430Z
642438eaa3adbc7142c3ca0f
/avatars/8deff70e0c93d259a42ee47f00a31e3e.svg
CharlesChen
user
CharlesChen2023
67e2a43d5116df47da357eed
Lin Gu
67e2a43d5116df47da357eee
Liang Li
67e2a43d5116df47da357eef
Chenggang Yan
67e2a43d5116df47da357ef0
Ying Fu
Frequency Dynamic Convolution for Dense Image Prediction
While Dynamic Convolution (DY-Conv) has shown promising performance by
enabling adaptive weight selection through multiple parallel weights combined
with an attention mechanism, the frequency response of these weights tends to
exhibit high similarity, resulting in high parameter costs but limited
adaptability. In this work, we introduce Frequency Dynamic Convolution
(FDConv), a novel approach that mitigates these limitations by learning a fixed
parameter budget in the Fourier domain. FDConv divides this budget into
frequency-based groups with disjoint Fourier indices, enabling the construction
of frequency-diverse weights without increasing the parameter cost. To further
enhance adaptability, we propose Kernel Spatial Modulation (KSM) and Frequency
Band Modulation (FBM). KSM dynamically adjusts the frequency response of each
filter at the spatial level, while FBM decomposes weights into distinct
frequency bands in the frequency domain and modulates them dynamically based on
local content. Extensive experiments on object detection, segmentation, and
classification validate the effectiveness of FDConv. We demonstrate that when
applied to ResNet-50, FDConv achieves superior performance with a modest
increase of +3.6M parameters, outperforming previous methods that require
substantial increases in parameter budgets (e.g., CondConv +90M, KW +76.5M).
Moreover, FDConv seamlessly integrates into a variety of architectures,
including ConvNeXt, Swin-Transformer, offering a flexible and efficient
solution for modern vision tasks. The code is made publicly available at
https://github.com/Linwei-Chen/FDConv.
FDConv improves performance in modern vision tasks by learning frequency-diverse weights with a fixed parameter budget and integrating modulation techniques in both spatial and frequency domains.
Dynamic Convolution DY-Conv Frequency Dynamic Convolution FDConv Fourier domain frequency-based groups Kernel Spatial Modulation KSM Frequency Band Modulation FBM object detection segmentation classification ResNet-50 ConvNeXt Swin-Transformer