RoCo-Sim: Enhancing Roadside Collaborative Perception through Foreground
  Simulation
Roadside Collaborative Perception refers to a system where multiple roadside
units collaborate to pool their perceptual data, assisting vehicles in
enhancing their environmental awareness. Existing roadside perception methods
concentrate on model design but overlook data issues like calibration errors,
sparse information, and multi-view consistency, leading to poor performance on
recent published datasets. To significantly enhance roadside collaborative
perception and address critical data issues, we present the first simulation
framework RoCo-Sim for road-side collaborative perception. RoCo-Sim is capable
of generating diverse, multi-view consistent simulated roadside data through
dynamic foreground editing and full-scene style transfer of a single image.
RoCo-Sim consists of four components: (1) Camera Extrinsic Optimization ensures
accurate 3D to 2D projection for roadside cameras; (2) A novel Multi-View
Occlusion-Aware Sampler (MOAS) determines the placement of diverse digital
assets within 3D space; (3) DepthSAM innovatively models foreground-background
relationships from single-frame fixed-view images, ensuring multi-view
consistency of foreground; and (4) Scalable Post-Processing Toolkit generates
more realistic and enriched scenes through style transfer and other
enhancements. RoCo-Sim significantly improves roadside 3D object detection,
outperforming SOTA methods by 83.74 on Rcooper-Intersection and 83.12 on
TUMTraf-V2X for AP70. RoCo-Sim fills a critical gap in roadside perception
simulation. Code and pre-trained models will be released soon:
https://github.com/duyuwen-duen/RoCo-Sim
67d9638d92e48ed07860ecee
Yuwen Du
claimed_verified
2025-03-18T14:57:49.908Z
67934b85c67af4a116b5594b
/avatars/076cf0803b50e1ab54e5ba4f8f2a8e44.svg
yuwendu
user
yuwendu
67d9638d92e48ed07860ecef
Anning Hu
admin_assigned
2025-03-19T13:44:46.862Z
6622a19fabae16393f24a3ff
/avatars/5578f13a000252d906455fc6361a4fa0.svg
Anning Hu
user
PathfinderToT
67d9638d92e48ed07860ecf0
Zichen Chao
67d9638d92e48ed07860ecf1
Yifan Lu
admin_assigned
2025-03-19T13:45:07.177Z
6434b8aeea46c009904ce8cb
https://cdn-avatars.huggingface.co/v1/production/uploads/6434b8aeea46c009904ce8cb/AiXL-BJ4KFSPbaorKPHNL.jpeg
Yifan Lu
user
yifanlu
67d9638d92e48ed07860ecf2
Junhao Ge
admin_assigned
2025-03-19T13:45:18.493Z
664aebb7cf5fd472d7c37b2f
/avatars/59b44b608a27269ab680cafa29a50f18.svg
Junhao Ge
user
Cancaries
67d9638d92e48ed07860ecf3
Genjia Liu
admin_assigned
2025-03-19T13:45:24.545Z
634187b38d8089ebaef3dc18
/avatars/a225077ee00d34cd4c084e3e347ce047.svg
Genjia Liu
user
gjliu
67d9638d92e48ed07860ecf4
Weitao Wu
admin_assigned
2025-03-19T13:45:39.488Z
65bc8587262a04f94c38808f
/avatars/9478bb68e931edc9b0eba86807de5579.svg
Wuweitao
user
VictorWuweitao
67d9638d92e48ed07860ecf5
Lanjun Wang
admin_assigned
2025-03-19T13:45:46.738Z
646ec1c65d05113e4327d900
/avatars/02b02a885cac821430bf4916e91e626e.svg
Lanjun Wang
user
lanzerwang
67d9638d92e48ed07860ecf6
Siheng Chen
admin_assigned
2025-03-19T13:45:52.526Z
65257545b017be1fc1915364
/avatars/9bffd3fb567d2fa1e5c3546d77560b43.svg
Siheng Chen
user
sihengchen
RoCo-Sim: Enhancing Roadside Collaborative Perception through Foreground
  Simulation
Roadside Collaborative Perception refers to a system where multiple roadside
units collaborate to pool their perceptual data, assisting vehicles in
enhancing their environmental awareness. Existing roadside perception methods
concentrate on model design but overlook data issues like calibration errors,
sparse information, and multi-view consistency, leading to poor performance on
recent published datasets. To significantly enhance roadside collaborative
perception and address critical data issues, we present the first simulation
framework RoCo-Sim for road-side collaborative perception. RoCo-Sim is capable
of generating diverse, multi-view consistent simulated roadside data through
dynamic foreground editing and full-scene style transfer of a single image.
RoCo-Sim consists of four components: (1) Camera Extrinsic Optimization ensures
accurate 3D to 2D projection for roadside cameras; (2) A novel Multi-View
Occlusion-Aware Sampler (MOAS) determines the placement of diverse digital
assets within 3D space; (3) DepthSAM innovatively models foreground-background
relationships from single-frame fixed-view images, ensuring multi-view
consistency of foreground; and (4) Scalable Post-Processing Toolkit generates
more realistic and enriched scenes through style transfer and other
enhancements. RoCo-Sim significantly improves roadside 3D object detection,
outperforming SOTA methods by 83.74 on Rcooper-Intersection and 83.12 on
TUMTraf-V2X for AP70. RoCo-Sim fills a critical gap in roadside perception
simulation. Code and pre-trained models will be released soon:
https://github.com/duyuwen-duen/RoCo-Sim
RoCo-Sim is a simulation framework for roadside collaborative perception that generates diverse, multi-view consistent data to improve object detection performance.
Camera Extrinsic Optimization Multi-View Occlusion-Aware Sampler DepthSAM Scalable Post-Processing Toolkit 3D object detection AP70 Rcooper-Intersection TUMTraf-V2X