The ObjectFolder Benchmark: Multisensory Learning with Neural and Real
  Objects
We introduce the ObjectFolder Benchmark, a benchmark suite of 10 tasks for
multisensory object-centric learning, centered around object recognition,
reconstruction, and manipulation with sight, sound, and touch. We also
introduce the ObjectFolder Real dataset, including the multisensory
measurements for 100 real-world household objects, building upon a newly
designed pipeline for collecting the 3D meshes, videos, impact sounds, and
tactile readings of real-world objects. We conduct systematic benchmarking on
both the 1,000 multisensory neural objects from ObjectFolder, and the real
multisensory data from ObjectFolder Real. Our results demonstrate the
importance of multisensory perception and reveal the respective roles of
vision, audio, and touch for different object-centric learning tasks. By
publicly releasing our dataset and benchmark suite, we hope to catalyze and
enable new research in multisensory object-centric learning in computer vision,
robotics, and beyond. Project page: https://objectfolder.stanford.edu
64798ed74be04c76ce451b44
Ruohan Gao
64798ed74be04c76ce451b45
Yiming Dou
admin_assigned
2023-06-05T12:12:38.085Z
63a29eb3f30c464227919797
/avatars/6eb9332ab57ea36fd0d670e06983a319.svg
Yiming Dou
user
ymdou
64798ed74be04c76ce451b46
Hao Li
64798ed74be04c76ce451b47
Tanmay Agarwal
claimed_verified
2023-06-02T12:46:37.789Z
6479aa2d9763c4ef86ae0ff0
/avatars/f8ff4a1f55fb9dd17e6527b2f5bb76a6.svg
Tanmay Agarwal
user
tanmayxagarwal
64798ed74be04c76ce451b48
Jeannette Bohg
claimed_verified
2024-12-18T09:55:42.915Z
676272a59865895e253ceee5
https://cdn-avatars.huggingface.co/v1/production/uploads/676272a59865895e253ceee5/Nz51Y-G49SD19lQEjH6Yd.jpeg
Jeannette Bohg
user
jbohg
64798ed74be04c76ce451b49
Yunzhu Li
claimed_verified
2023-07-04T19:17:35.883Z
64a46562c641afb468a3dc04
/avatars/bc54e5c13f657251b05bf2725086e56d.svg
Yunzhu Li
user
yunzhuli
64798ed74be04c76ce451b4a
Li Fei-Fei
64798ed74be04c76ce451b4b
Jiajun Wu
claimed_verified
2023-06-05T10:06:28.511Z
647b7b8345616c3de3627863
/avatars/8196364bffed71086ba9a0a4a62027d3.svg
Jiajun Wu
user
jiajunwu
The ObjectFolder Benchmark: Multisensory Learning with Neural and Real
  Objects
We introduce the ObjectFolder Benchmark, a benchmark suite of 10 tasks for
multisensory object-centric learning, centered around object recognition,
reconstruction, and manipulation with sight, sound, and touch. We also
introduce the ObjectFolder Real dataset, including the multisensory
measurements for 100 real-world household objects, building upon a newly
designed pipeline for collecting the 3D meshes, videos, impact sounds, and
tactile readings of real-world objects. We conduct systematic benchmarking on
both the 1,000 multisensory neural objects from ObjectFolder, and the real
multisensory data from ObjectFolder Real. Our results demonstrate the
importance of multisensory perception and reveal the respective roles of
vision, audio, and touch for different object-centric learning tasks. By
publicly releasing our dataset and benchmark suite, we hope to catalyze and
enable new research in multisensory object-centric learning in computer vision,
robotics, and beyond. Project page: https://objectfolder.stanford.edu
A benchmark suite and real dataset for multisensory object-centric learning reveal the importance of vision, audio, and touch in various object tasks.
multisensory object-centric learning object recognition reconstruction manipulation ObjectFolder Benchmark ObjectFolder Real 3D meshes videos impact sounds tactile readings