Tracking Any Object Amodally
Amodal perception, the ability to comprehend complete object structures from
partial visibility, is a fundamental skill, even for infants. Its significance
extends to applications like autonomous driving, where a clear understanding of
heavily occluded objects is essential. However, modern detection and tracking
algorithms often overlook this critical capability, perhaps due to the
prevalence of modal annotations in most datasets. To address the scarcity of
amodal data, we introduce the TAO-Amodal benchmark, featuring 880 diverse
categories in thousands of video sequences. Our dataset includes amodal and
modal bounding boxes for visible and occluded objects, including objects that
are partially out-of-frame. To enhance amodal tracking with object permanence,
we leverage a lightweight plug-in module, the amodal expander, to transform
standard, modal trackers into amodal ones through fine-tuning on a few hundred
video sequences with data augmentation. We achieve a 3.3\% and 1.6\%
improvement on the detection and tracking of occluded objects on TAO-Amodal.
When evaluated on people, our method produces dramatic improvements of 2x
compared to state-of-the-art modal baselines.
6582627192b5a9664de9ffe5
Cheng-Yen Hsieh
admin_assigned
2023-12-20T09:20:42.742Z
649d72f62634f8711f32b89c
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/z7OZ0L_yl8UFzWKAWebNW.jpeg
Cheng-Yen Hsieh
user
chengyenhsieh
6582627192b5a9664de9ffe6
Tarasha Khurana
6582627192b5a9664de9ffe7
Achal Dave
admin_assigned
2023-12-20T09:21:23.333Z
6488e29b78570cedad71eb2d
/avatars/0d8bb52c1e2be8d6aa7c732cf92fc5a9.svg
Achal Dave
user
achal-tri
6582627192b5a9664de9ffe8
Deva Ramanan
admin_assigned
2023-12-20T09:21:29.842Z
6337151b0267ebcf02640eb6
/avatars/14a723cafc5587043bdfb19304fc202d.svg
Deva Ramanan
user
devakramanan
Tracking Any Object Amodally
Amodal perception, the ability to comprehend complete object structures from
partial visibility, is a fundamental skill, even for infants. Its significance
extends to applications like autonomous driving, where a clear understanding of
heavily occluded objects is essential. However, modern detection and tracking
algorithms often overlook this critical capability, perhaps due to the
prevalence of modal annotations in most datasets. To address the scarcity of
amodal data, we introduce the TAO-Amodal benchmark, featuring 880 diverse
categories in thousands of video sequences. Our dataset includes amodal and
modal bounding boxes for visible and occluded objects, including objects that
are partially out-of-frame. To enhance amodal tracking with object permanence,
we leverage a lightweight plug-in module, the amodal expander, to transform
standard, modal trackers into amodal ones through fine-tuning on a few hundred
video sequences with data augmentation. We achieve a 3.3\% and 1.6\%
improvement on the detection and tracking of occluded objects on TAO-Amodal.
When evaluated on people, our method produces dramatic improvements of 2x
compared to state-of-the-art modal baselines.
A new TAO-Amodal benchmark and an amodal expander module improve amodal detection and tracking of occluded objects in video sequences.
amodal perception TAO-Amodal benchmark amodal bounding boxes modal bounding boxes object permanence amodal expander fine-tuning data augmentation