Temporally-consistent 3D Reconstruction of Birds
This paper deals with 3D reconstruction of seabirds which recently came into
focus of environmental scientists as valuable bio-indicators for environmental
change. Such 3D information is beneficial for analyzing the bird's behavior and
physiological shape, for example by tracking motion, shape, and appearance
changes. From a computer vision perspective birds are especially challenging
due to their rapid and oftentimes non-rigid motions. We propose an approach to
reconstruct the 3D pose and shape from monocular videos of a specific breed of
seabird - the common murre. Our approach comprises a full pipeline of
detection, tracking, segmentation, and temporally consistent 3D reconstruction.
Additionally, we propose a temporal loss that extends current single-image 3D
bird pose estimators to the temporal domain. Moreover, we provide a real-world
dataset of 10000 frames of video observations on average capture nine birds
simultaneously, comprising a large variety of motions and interactions,
including a smaller test set with bird-specific keypoint labels. Using our
temporal optimization, we achieve state-of-the-art performance for the
challenging sequences in our dataset.
66cf099c59013592dde34887
Johannes Hägerlind
66cf099c59013592dde34888
Jonas Hentati-Sundberg
extracted_confirmed
2024-08-28T12:35:19.146Z
642085fb69a2c293387fc661
/avatars/72935ec35b43fb01ab1e6d0b03ba520f.svg
Jonas Hentati Sundberg
user
jonhensun
66cf099c59013592dde34889
Bastian Wandt
Temporally-consistent 3D Reconstruction of Birds
This paper deals with 3D reconstruction of seabirds which recently came into
focus of environmental scientists as valuable bio-indicators for environmental
change. Such 3D information is beneficial for analyzing the bird's behavior and
physiological shape, for example by tracking motion, shape, and appearance
changes. From a computer vision perspective birds are especially challenging
due to their rapid and oftentimes non-rigid motions. We propose an approach to
reconstruct the 3D pose and shape from monocular videos of a specific breed of
seabird - the common murre. Our approach comprises a full pipeline of
detection, tracking, segmentation, and temporally consistent 3D reconstruction.
Additionally, we propose a temporal loss that extends current single-image 3D
bird pose estimators to the temporal domain. Moreover, we provide a real-world
dataset of 10000 frames of video observations on average capture nine birds
simultaneously, comprising a large variety of motions and interactions,
including a smaller test set with bird-specific keypoint labels. Using our
temporal optimization, we achieve state-of-the-art performance for the
challenging sequences in our dataset.
A pipeline for monocular 3D pose and shape reconstruction of seabirds includes detection, tracking, segmentation, and temporal consistency with a custom temporal loss, achieving state-of-the-art performance on challenging real-world video sequences.
3D reconstruction computer vision non-rigid motions detection tracking segmentation temporally consistent temporal loss single-image 3D bird pose estimators real-world dataset bird-specific keypoint labels