SMITE: Segment Me In TimE
Segmenting an object in a video presents significant challenges. Each pixel
must be accurately labelled, and these labels must remain consistent across
frames. The difficulty increases when the segmentation is with arbitrary
granularity, meaning the number of segments can vary arbitrarily, and masks are
defined based on only one or a few sample images. In this paper, we address
this issue by employing a pre-trained text to image diffusion model
supplemented with an additional tracking mechanism. We demonstrate that our
approach can effectively manage various segmentation scenarios and outperforms
state-of-the-art alternatives.
671b01ae8750e04558bb91d7
Amirhossein Alimohammadi
claimed_verified
2024-10-25T09:29:55.956Z
650e3abcae507a2c7c847baa
/avatars/a138bc6c9ba80a486fb36f4e1aa16b33.svg
Amirhossein Alimohammadi
user
Amirhossein-Alimohammadi
671b01ae8750e04558bb91d8
Sauradip Nag
claimed_verified
2024-10-25T09:29:53.932Z
6399ab3296ce14c5dcf4ccbf
/avatars/89aeedc96f73d76f7a6da96454f40fd2.svg
Sauradip Nag
user
sauradip
671b01ae8750e04558bb91d9
Saeid Asgari Taghanaki
671b01ae8750e04558bb91da
Andrea Tagliasacchi
671b01ae8750e04558bb91db
Ghassan Hamarneh
671b01ae8750e04558bb91dc
Ali Mahdavi Amiri
SMITE: Segment Me In TimE
Segmenting an object in a video presents significant challenges. Each pixel
must be accurately labelled, and these labels must remain consistent across
frames. The difficulty increases when the segmentation is with arbitrary
granularity, meaning the number of segments can vary arbitrarily, and masks are
defined based on only one or a few sample images. In this paper, we address
this issue by employing a pre-trained text to image diffusion model
supplemented with an additional tracking mechanism. We demonstrate that our
approach can effectively manage various segmentation scenarios and outperforms
state-of-the-art alternatives.
A pre-trained text-to-image diffusion model with tracking mechanism outperforms existing methods in object segmentation across video frames with arbitrary granularity.
diffusion model tracking mechanism object segmentation video frames arbitrary granularity