Degeneracy in Self-Calibration Revisited and
a Deep Learning Solution for Uncalibrated SLAM

Bingbing Zhuang1 Quoc-Huy Tran2 Gim Hee Lee1 Loong Fah Cheong1 Manmohan Chandraker2,3
1National University of Singapore 2NEC Labs America 3University of California, San Diego
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019 [Oral]
From a set of calibrated images with known camera intrinsics and no radial distortion, we generate synthetic uncalibrated images for training our network by randomly adjusting camera intrinsics and randomly adding radial distortion. At testing, given a single real uncalibrated image, our network predicts accurate camera parameters, which are used for producing the calibrated image.

Abstract

Self-calibration of camera intrinsics and radial distortion has a long history of research in the computer vision community. However, it remains rare to see real applications of such techniques to modern Simultaneous Localization And Mapping (SLAM) systems, especially in driving scenarios. In this paper, we revisit the geometric approach to this problem, and provide a theoretical proof that explicitly shows the ambiguity between radial distortion and scene depth when two-view geometry is used to self-calibrate the radial distortion. In view of such geometric degeneracy, we propose a learning approach that trains a convolutional neural network (CNN) on a large amount of synthetic data. We demonstrate the utility of our proposed method by applying it as a checkerboard-free calibration tool for SLAM, achieving comparable or superior performance to previous learning and hand-crafted methods.

Paper

Degeneracy in Self-Calibration Revisited and a Deep Learning Solution for Uncalibrated SLAM
Bingbing Zhuang, Quoc-Huy Tran, Gim Hee Lee, Loong-Fah Cheong, Manmohan Chandraker
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019 [Oral]
[Preprint]  [Bibtex]

Geometric Approach vs. Learning Approach
in Near-Degenerate Setting

Radial distortion estimation by geometric and learning approaches. (a) Radial distortion estimates. (b) PDF of radial distortion estimates.

Single-View Self-Calibration Results

Qualitative results of single-view camera self-calibration on the KITTI Raw test sequence.
Qualitative results of single-view camera self-calibration on YouTube test sequences. (a)-(b) Sequence 1. (c)-(d) Sequence 2. (e)-(f) Sequence 3.

Uncalibrated SLAM Results

Qualitative results of uncalibrated SLAM on the first 400 frames of the KITTI Raw test sequence. Red points denote the latest local map.
Qualitative results of uncalibrated SLAM on the entire KITTI Raw test sequence. Red points denote the latest local map.
Quantitative comparisons in terms of median RMSE of keyframe trajectory over 5 runs on the entire KITTI Raw test sequence. X denotes that `HandCrafted+OVP' breaks ORB-SLAM in all 5 runs.
Qualitative results of uncalibrated SLAM on YouTube test sequences. Red points denote the latest local map.

Acknowledgements

We want to thank Pan Ji for joining initial discussions and helping implement the multi-task baseline. Part of this work was done during Bingbing Zhuang’s internship at NEC Labs America. This work is also partially supported by the Singapore PSF grant 1521200082 and MOE Tier 1 grant R-252-000-A65-114. This website template is inspired by this website.