SkillFormer: Unified Multi-View Video Understanding for Proficiency
  Estimation
Assessing human skill levels in complex activities is a challenging problem
with applications in sports, rehabilitation, and training. In this work, we
present SkillFormer, a parameter-efficient architecture for unified multi-view
proficiency estimation from egocentric and exocentric videos. Building on the
TimeSformer backbone, SkillFormer introduces a CrossViewFusion module that
fuses view-specific features using multi-head cross-attention, learnable
gating, and adaptive self-calibration. We leverage Low-Rank Adaptation to
fine-tune only a small subset of parameters, significantly reducing training
costs. In fact, when evaluated on the EgoExo4D dataset, SkillFormer achieves
state-of-the-art accuracy in multi-view settings while demonstrating remarkable
computational efficiency, using 4.5x fewer parameters and requiring 3.75x fewer
training epochs than prior baselines. It excels in multiple structured tasks,
confirming the value of multi-view integration for fine-grained skill
assessment.
68243bddd08d8e01109d5680
Edoardo Bianchi
extracted_confirmed
2025-05-14T06:45:06.471Z
622dc11fe27c88667db093fc
https://cdn-avatars.huggingface.co/v1/production/uploads/1667052350862-622dc11fe27c88667db093fc.jpeg
Edoardo Bianchi
user
EdBianchi
68243bddd08d8e01109d5681
Antonio Liotta
admin_assigned
2025-05-14T10:11:54.811Z
66f2ab691e8b23ab0af8436e
/avatars/161a26e9444a860128282e553a95641c.svg
Antonio Liotta
user
ucaclio
SkillFormer: Unified Multi-View Video Understanding for Proficiency
  Estimation
Assessing human skill levels in complex activities is a challenging problem
with applications in sports, rehabilitation, and training. In this work, we
present SkillFormer, a parameter-efficient architecture for unified multi-view
proficiency estimation from egocentric and exocentric videos. Building on the
TimeSformer backbone, SkillFormer introduces a CrossViewFusion module that
fuses view-specific features using multi-head cross-attention, learnable
gating, and adaptive self-calibration. We leverage Low-Rank Adaptation to
fine-tune only a small subset of parameters, significantly reducing training
costs. In fact, when evaluated on the EgoExo4D dataset, SkillFormer achieves
state-of-the-art accuracy in multi-view settings while demonstrating remarkable
computational efficiency, using 4.5x fewer parameters and requiring 3.75x fewer
training epochs than prior baselines. It excels in multiple structured tasks,
confirming the value of multi-view integration for fine-grained skill
assessment.
SkillFormer, a parameter-efficient architecture, uses the TimeSformer backbone with a CrossViewFusion module to achieve state-of-the-art accuracy in multi-view skill assessment from egocentric and exocentric videos.
TimeSformer CrossViewFusion multi-head cross-attention learnable gating adaptive self-calibration Low-Rank Adaptation EgoExo4D dataset