1National University of Singapore2NEC Labs America3University of California, San Diego
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019 [Oral]
From a set of calibrated images with known camera intrinsics and no radial distortion, we generate synthetic uncalibrated images for training our network by randomly adjusting camera intrinsics and randomly adding radial distortion. At testing, given a single real uncalibrated image, our network predicts accurate camera parameters, which are used for producing the calibrated image.
Self-calibration of camera intrinsics and radial distortion has a long history of research in the computer vision community. However, it remains rare to see real applications of such techniques to modern Simultaneous Localization And Mapping (SLAM) systems, especially in driving scenarios. In this paper, we revisit the geometric approach to this problem, and provide a theoretical proof that explicitly shows the ambiguity between radial distortion and scene depth when two-view geometry is used to self-calibrate the radial distortion. In view of such geometric degeneracy, we propose a learning approach that trains a convolutional neural network (CNN) on a large amount of synthetic data. We demonstrate the utility of our proposed method by applying it as a checkerboard-free calibration tool for SLAM, achieving comparable or superior performance to previous learning and hand-crafted methods.
Degeneracy in Self-Calibration Revisited and a Deep Learning Solution for Uncalibrated SLAM
Bingbing Zhuang, Quoc-Huy Tran, Gim Hee Lee, Loong-Fah Cheong, Manmohan Chandraker
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019 [Oral] [Preprint][Bibtex]
Geometric Approach vs. Learning Approach in Near-Degenerate Setting
Radial distortion estimation by geometric and learning approaches. (a) Radial distortion estimates. (b) PDF of radial distortion estimates.
Single-View Self-Calibration Results
Qualitative results of single-view camera self-calibration on the KITTI Raw test sequence.
Qualitative results of single-view camera self-calibration on YouTube test sequences. (a)-(b) Sequence 1. (c)-(d) Sequence 2. (e)-(f) Sequence 3.
Uncalibrated SLAM Results
Qualitative results of uncalibrated SLAM on the first 400 frames of the KITTI Raw test sequence. Red points denote the latest local map.
Qualitative results of uncalibrated SLAM on the entire KITTI Raw test sequence. Red points denote the latest local map.
Quantitative comparisons in terms of median RMSE of keyframe trajectory over 5 runs on the entire KITTI Raw test sequence. X denotes that `HandCrafted+OVP' breaks ORB-SLAM in all 5 runs.
Qualitative results of uncalibrated SLAM on YouTube test sequences. Red points denote the latest local map.
We want to thank Pan Ji for joining initial discussions and helping implement the multi-task baseline. Part of this work was done during Bingbing Zhuang’s internship at NEC Labs America. This work is also partially supported by the Singapore PSF grant 1521200082 and MOE Tier 1 grant R-252-000-A65-114. This website template is inspired by this website.