MEDIA ANALYTICS
PROJECTS
PEOPLE
PUBLICATIONS
PATENTS
Image Stitching and Rectification for Hand-Held Cameras
We derive a new differential homography that can account for the scanline-varying camera poses in rolling shutter (RS) cameras, and demonstrate its application to carry out RS-aware image stitching and rectification at one stroke. Despite the high complexity of RS geometry, we focus in this paper on a special yet common input: two consecutive frames from a video stream wherein the interframe motion is restricted from being arbitrarily large.
Collaborators: Bingbing Zhuang, Quoc-Huy Tran
(Top) Consecutive frames from a video captured by a hand-held rolling shutter (RS) camera are input to our method. (Bottom) Results of our RS-aware image stitching and rectification method are shown.
Project Site
Image Stitching and Rectification for Hand-Held Cameras Paper
Bingbing Zhuang & Quoc-Huy Tran
NEC Labs America
European Conference on Computer Vision (ECCV), 2020
[Download Paper PDF] [Supplementary Paper] [Supplementary Paper GitHub] [Bibtex]
Abstract
In this paper, we derive a new differential homography that can account for the scanline-varying camera poses in Rolling Shutter (RS) cameras, and demonstrate its application to carry out RS-aware image stitching and rectification at one stroke. Despite the high complexity of RS geometry, we focus in this paper on a special yet common input — two consecutive frames from a video stream, wherein the inter-frame motion is restricted from being arbitrarily large. This allows us to adopt simpler differential motion model, leading to a straightforward and practical minimal solver. To deal with non-planar scene and camera parallax in stitching, we further propose an RS-aware spatially-varying homogarphy field in the principle of As-Projective-As-Possible (APAP). We show superior performance over state-of-the-art methods both in RS image stitching and rectification, especially for images captured by hand-held shaking cameras.
Two-View Experiments
Qualitative comparison of image rectification and stitching on real RS images.
Quantitative evaluation under standard setting & further study.