EgoNight: Towards Egocentric Vision Understanding at Night with a
  Challenging Benchmark
Most existing benchmarks for egocentric vision understanding focus primarily
on daytime scenarios, overlooking the low-light conditions that are inevitable
in real-world applications. To investigate this gap, we present EgoNight, the
first comprehensive benchmark for nighttime egocentric vision, with visual
question answering (VQA) as the core task. A key feature of EgoNight is the
introduction of day-night aligned videos, which enhance night annotation
quality using the daytime data and reveal clear performance gaps between
lighting conditions. To achieve this, we collect both synthetic videos rendered
by Blender and real-world recordings, ensuring that scenes and actions are
visually and temporally aligned. Leveraging these paired videos, we construct
EgoNight-VQA, supported by a novel day-augmented night auto-labeling engine and
refinement through extensive human verification. Each QA pair is double-checked
by annotators for reliability. In total, EgoNight-VQA contains 3658 QA pairs
across 90 videos, spanning 12 diverse QA types, with more than 300 hours of
human work. Evaluations of state-of-the-art multimodal large language models
(MLLMs) reveal substantial performance drops when transferring from day to
night, underscoring the challenges of reasoning under low-light conditions.
Beyond VQA, EgoNight also introduces two auxiliary tasks, day-night
correspondence retrieval and egocentric depth estimation at night, that further
explore the boundaries of existing models. We believe EgoNight-VQA provides a
strong foundation for advancing application-driven egocentric vision research
and for developing models that generalize across illumination domains. All the
data and code will be made available upon acceptance.
68e5c2e8975ac4c405ef20bd
Deheng Zhang
claimed_verified
2025-10-08T11:49:07.341Z
6814b27fdc0f8971ef7cd574
/avatars/590ed5d0f2f9933c90df71cc69b9401e.svg
Deheng Zhang
user
dehezhang2
68e5c2e8975ac4c405ef20be
Yuqian Fu
68e5c2e8975ac4c405ef20bf
Runyi Yang
68e5c2e8975ac4c405ef20c0
Yang Miao
68e5c2e8975ac4c405ef20c1
Tianwen Qian
68e5c2e8975ac4c405ef20c2
Xu Zheng
68e5c2e8975ac4c405ef20c3
Guolei Sun
68e5c2e8975ac4c405ef20c4
Ajad Chhatkuli
68e5c2e8975ac4c405ef20c5
Xuanjing Huang
68e5c2e8975ac4c405ef20c6
Yu-Gang Jiang
68e5c2e8975ac4c405ef20c7
Luc Van Gool
68e5c2e8975ac4c405ef20c8
Danda Pani Paudel
EgoNight: Towards Egocentric Vision Understanding at Night with a
  Challenging Benchmark
Most existing benchmarks for egocentric vision understanding focus primarily
on daytime scenarios, overlooking the low-light conditions that are inevitable
in real-world applications. To investigate this gap, we present EgoNight, the
first comprehensive benchmark for nighttime egocentric vision, with visual
question answering (VQA) as the core task. A key feature of EgoNight is the
introduction of day-night aligned videos, which enhance night annotation
quality using the daytime data and reveal clear performance gaps between
lighting conditions. To achieve this, we collect both synthetic videos rendered
by Blender and real-world recordings, ensuring that scenes and actions are
visually and temporally aligned. Leveraging these paired videos, we construct
EgoNight-VQA, supported by a novel day-augmented night auto-labeling engine and
refinement through extensive human verification. Each QA pair is double-checked
by annotators for reliability. In total, EgoNight-VQA contains 3658 QA pairs
across 90 videos, spanning 12 diverse QA types, with more than 300 hours of
human work. Evaluations of state-of-the-art multimodal large language models
(MLLMs) reveal substantial performance drops when transferring from day to
night, underscoring the challenges of reasoning under low-light conditions.
Beyond VQA, EgoNight also introduces two auxiliary tasks, day-night
correspondence retrieval and egocentric depth estimation at night, that further
explore the boundaries of existing models. We believe EgoNight-VQA provides a
strong foundation for advancing application-driven egocentric vision research
and for developing models that generalize across illumination domains. All the
data and code will be made available upon acceptance.
EgoNight is a comprehensive benchmark for nighttime egocentric vision, focusing on visual question answering and revealing performance gaps between day and night conditions for multimodal large language models.
egocentric vision visual question answering day-night aligned videos synthetic videos real-world recordings day-augmented night auto-labeling multimodal large language models day-night correspondence retrieval egocentric depth estimation