Cockatiel: Ensembling Synthetic and Human Preferenced Training for
  Detailed Video Caption
Video Detailed Captioning (VDC) is a crucial task for vision-language
bridging, enabling fine-grained descriptions of complex video content. In this
paper, we first comprehensively benchmark current state-of-the-art approaches
and systematically identified two critical limitations: biased capability
towards specific captioning aspect and misalignment with human preferences. To
address these deficiencies, we propose Cockatiel, a novel three-stage training
pipeline that ensembles synthetic and human-aligned training for improving VDC
performance. In the first stage, we derive a scorer from a meticulously
annotated dataset to select synthetic captions high-performing on certain
fine-grained video-caption alignment and human-preferred while disregarding
others. Then, we train Cockatiel-13B, using this curated dataset to infuse it
with assembled model strengths and human preferences. Finally, we further
distill Cockatiel-8B from Cockatiel-13B for the ease of usage. Extensive
quantitative and qualitative experiments reflect the effectiveness of our
method, as we not only set new state-of-the-art performance on VDCSCORE in a
dimension-balanced way but also surpass leading alternatives on human
preference by a large margin as depicted by the human evaluation results.
67d2bd340860f2d7ff10e3dc
Luozheng Qin
admin_assigned
2025-03-17T08:56:11.189Z
66a9b3533d417b0baa9220a6
/avatars/adc372bd24df1d3bf43258833411e8af.svg
Luozheng Qin
user
Fr0zencr4nE
67d2bd340860f2d7ff10e3dd
Zhiyu Tan
67d2bd340860f2d7ff10e3de
Mengping Yang
admin_assigned
2025-03-17T08:56:25.200Z
6304d630dae2eb7d084148c7
/avatars/7d7a6ca99334bdae3ed1752ff40a8d94.svg
mengping yang
user
Kobeshegu
67d2bd340860f2d7ff10e3df
Xiaomeng Yang
admin_assigned
2025-03-17T08:56:33.077Z
658ea92268d0b7633176b4ed
/avatars/40173c9126dccfe78bc46b12c6ced8c8.svg
xiaomeng yang
user
xiaomengyang
67d2bd340860f2d7ff10e3e0
Hao Li
Cockatiel: Ensembling Synthetic and Human Preferenced Training for
  Detailed Video Caption
Video Detailed Captioning (VDC) is a crucial task for vision-language
bridging, enabling fine-grained descriptions of complex video content. In this
paper, we first comprehensively benchmark current state-of-the-art approaches
and systematically identified two critical limitations: biased capability
towards specific captioning aspect and misalignment with human preferences. To
address these deficiencies, we propose Cockatiel, a novel three-stage training
pipeline that ensembles synthetic and human-aligned training for improving VDC
performance. In the first stage, we derive a scorer from a meticulously
annotated dataset to select synthetic captions high-performing on certain
fine-grained video-caption alignment and human-preferred while disregarding
others. Then, we train Cockatiel-13B, using this curated dataset to infuse it
with assembled model strengths and human preferences. Finally, we further
distill Cockatiel-8B from Cockatiel-13B for the ease of usage. Extensive
quantitative and qualitative experiments reflect the effectiveness of our
method, as we not only set new state-of-the-art performance on VDCSCORE in a
dimension-balanced way but also surpass leading alternatives on human
preference by a large margin as depicted by the human evaluation results.
A three-stage training pipeline combining synthetic and human-aligned data improves Video Detailed Captioning performance, outperforming existing methods in both quantitative metrics and human preference.
video detailed captioning vision-language bridging captioning aspect human preferences three-stage training pipeline synthetic captions human-aligned training VDCSCORE model strengths human evaluation