Thinking with Camera: A Unified Multimodal Model for Camera-Centric
  Understanding and Generation
Camera-centric understanding and generation are two cornerstones of spatial
intelligence, yet they are typically studied in isolation. We present Puffin, a
unified camera-centric multimodal model that extends spatial awareness along
the camera dimension. Puffin integrates language regression and diffusion-based
generation to interpret and create scenes from arbitrary viewpoints. To bridge
the modality gap between cameras and vision-language, we introduce a novel
paradigm that treats camera as language, enabling thinking with camera. This
guides the model to align spatially grounded visual cues with photographic
terminology while reasoning across geometric context. Puffin is trained on
Puffin-4M, a large-scale dataset of 4 million vision-language-camera triplets.
We incorporate both global camera parameters and pixel-wise camera maps,
yielding flexible and reliable spatial generation. Experiments demonstrate
Puffin superior performance over specialized models for camera-centric
generation and understanding. With instruction tuning, Puffin generalizes to
diverse cross-view tasks such as spatial imagination, world exploration, and
photography guidance. We will release the code, models, dataset pipeline, and
benchmark to advance multimodal spatial intelligence research.
68ec5edfcd07fb414898c93e
Kang Liao
claimed_verified
2025-10-13T10:05:58.256Z
65bc98383b879593a5a2f5e5
https://cdn-avatars.huggingface.co/v1/production/uploads/65bc98383b879593a5a2f5e5/p2ZtoTFN6tW-QkcPJf7YT.jpeg
Kang Liao
user
KangLiao
68ec5edfcd07fb414898c93f
Size Wu
68ec5edfcd07fb414898c940
Zhonghua Wu
68ec5edfcd07fb414898c941
Linyi Jin
68ec5edfcd07fb414898c942
Chao Wang
68ec5edfcd07fb414898c943
Yikai Wang
claimed_verified
2025-10-13T10:06:01.008Z
63463bc4547c70e4b7d3009f
/avatars/6e5350fd998f0a7a4143d7504218164a.svg
Yikai Wang
user
yikaiwang
68ec5edfcd07fb414898c944
Fei Wang
68ec5edfcd07fb414898c945
Wei Li
68ec5edfcd07fb414898c946
Chen Change Loy
Thinking with Camera: A Unified Multimodal Model for Camera-Centric
  Understanding and Generation
Camera-centric understanding and generation are two cornerstones of spatial
intelligence, yet they are typically studied in isolation. We present Puffin, a
unified camera-centric multimodal model that extends spatial awareness along
the camera dimension. Puffin integrates language regression and diffusion-based
generation to interpret and create scenes from arbitrary viewpoints. To bridge
the modality gap between cameras and vision-language, we introduce a novel
paradigm that treats camera as language, enabling thinking with camera. This
guides the model to align spatially grounded visual cues with photographic
terminology while reasoning across geometric context. Puffin is trained on
Puffin-4M, a large-scale dataset of 4 million vision-language-camera triplets.
We incorporate both global camera parameters and pixel-wise camera maps,
yielding flexible and reliable spatial generation. Experiments demonstrate
Puffin superior performance over specialized models for camera-centric
generation and understanding. With instruction tuning, Puffin generalizes to
diverse cross-view tasks such as spatial imagination, world exploration, and
photography guidance. We will release the code, models, dataset pipeline, and
benchmark to advance multimodal spatial intelligence research.
Puffin, a unified multimodal model, integrates language regression and diffusion-based generation to enhance camera-centric spatial understanding and generation by treating camera parameters as language.
language regression diffusion-based generation camera-centric multimodal model spatial awareness vision-language camera as language geometric context Puffin-4M global camera parameters pixel-wise camera maps spatial generation instruction tuning spatial imagination world exploration photography guidance