Efficient Hybrid Zoom using Camera Fusion on Mobile Phones
DSLR cameras can achieve multiple zoom levels via shifting lens distances or
swapping lens types. However, these techniques are not possible on smartphone
devices due to space constraints. Most smartphone manufacturers adopt a hybrid
zoom system: commonly a Wide (W) camera at a low zoom level and a Telephoto (T)
camera at a high zoom level. To simulate zoom levels between W and T, these
systems crop and digitally upsample images from W, leading to significant
detail loss. In this paper, we propose an efficient system for hybrid zoom
super-resolution on mobile devices, which captures a synchronous pair of W and
T shots and leverages machine learning models to align and transfer details
from T to W. We further develop an adaptive blending method that accounts for
depth-of-field mismatches, scene occlusion, flow uncertainty, and alignment
errors. To minimize the domain gap, we design a dual-phone camera rig to
capture real-world inputs and ground-truths for supervised training. Our method
generates a 12-megapixel image in 500ms on a mobile platform and compares
favorably against state-of-the-art methods under extensive evaluation on
real-world scenarios.
65963006d61d4cbc1be7d058
Xiaotong Wu
65963006d61d4cbc1be7d059
Wei-Sheng Lai
65963006d61d4cbc1be7d05a
YiChang Shih
65963006d61d4cbc1be7d05b
Charles Herrmann
65963006d61d4cbc1be7d05c
Michael Krainin
65963006d61d4cbc1be7d05d
Deqing Sun
65963006d61d4cbc1be7d05e
Chia-Kai Liang
Efficient Hybrid Zoom using Camera Fusion on Mobile Phones
DSLR cameras can achieve multiple zoom levels via shifting lens distances or
swapping lens types. However, these techniques are not possible on smartphone
devices due to space constraints. Most smartphone manufacturers adopt a hybrid
zoom system: commonly a Wide (W) camera at a low zoom level and a Telephoto (T)
camera at a high zoom level. To simulate zoom levels between W and T, these
systems crop and digitally upsample images from W, leading to significant
detail loss. In this paper, we propose an efficient system for hybrid zoom
super-resolution on mobile devices, which captures a synchronous pair of W and
T shots and leverages machine learning models to align and transfer details
from T to W. We further develop an adaptive blending method that accounts for
depth-of-field mismatches, scene occlusion, flow uncertainty, and alignment
errors. To minimize the domain gap, we design a dual-phone camera rig to
capture real-world inputs and ground-truths for supervised training. Our method
generates a 12-megapixel image in 500ms on a mobile platform and compares
favorably against state-of-the-art methods under extensive evaluation on
real-world scenarios.
An efficient hybrid zoom super-resolution system for mobile devices uses machine learning to align and transfer details from telephoto to wide shots, addressing depth-of-field mismatches and other issues.
hybrid zoom super-resolution machine learning align transfer details depth-of-field mismatches scene occlusion flow uncertainty alignment errors dual-phone camera rig