Publication Date: 2/22/2018
Event: IEEE Transactions on Pattern Analysis and Machine Intelligence
Reference: Vol. 40, No. 3, pp 740-754
Authors: Ting-Chun Wang, University of California, Berkeley; Manmohan Chandraker, NEC Laboratories America, Inc.; Alexei A. Efros, University of California, Berkeley; Ravi Ramamoorthi, University of California, San Diego
Abstract: Light-field cameras have recently emerged as a powerful tool for one-shot passive 3D shape capture. However, obtaining the shape of glossy objects like metals or plastics remains challenging, since standard Lambertian cues like photo-consistency cannot be easily applied. In this paper, we derive a spatially-varying (SV)BRDF-invariant theory for recovering 3D shape and reflectance from light-field cameras. Our key theoretical insight is a novel analysis of diffuse plus single-lobe SVBRDFs under a light-field setup. We show that, although direct shape recovery is not possible, an equation relating depths and normals can still be derived. Using this equation, we then propose using a polynomial (quadratic) shape prior to resolve the shape ambiguity. Once shape is estimated, we also recover the reflectance. We present extensive synthetic data on the entire MERL BRDF dataset, as well as a number of real examples to validate the theory, where we simultaneously recover shape and BRDFs from a single image taken with a Lytro Illum camera.
Publication Link: https://ieeexplore.ieee.org/document/7875163