Abstract

We propose a new multiplanar superstructure for unified real-time processing of RGB-D data. Modern RGB-D sensors are widely used for indoor 3D capture, with applications ranging from modeling to robotics, through augmented reality. Nevertheless, their use is limited by their low resolution, with frames often corrupted with noise, missing data and temporal inconsistencies. Our approach, named Proxy Clouds, consists in generating and updating through time a single set of compact local statistics parameterized over detected planar proxies, which are fed from raw RGB-D data. Proxy Clouds provide several processing primitives, which improve the quality of the RGB-D stream on-the-fly or lighten further operations. Experimental results confirm that our light weight analysis framework copes well with embedded execution as well as moderate memory and computational capabilities compared to state-of-the-art methods. Processing of RGB-D data with Proxy Clouds includes noise and temporal flickering removal, hole filling and resampling. As a substitute of the observed scene, our proxy cloud can additionally be applied to compression and scene reconstruction. We present experiments performed with our framework in indoor scenes of different natures within a recent open RGB-D dataset.

Keywords: RGB-D stream, 3D geometric primitives, data reinforcement, depth improvement, online processing, scene reconstruction

Video

Downloads

• ECCV 2018 Full Paper (CVF link)
• SIGGRAPH 2017 Talk Abstract (ACM link)

Bibtex

Main reference:

@inproceedings{Kaiser:2018:PC,
 author = {Adrien Kaiser, Alonso Ybanez and Tamy Boubekeur},
 title = {Proxy Clouds for Live RGB-D Stream Processing and Consolidation},
 booktitle = {Proceedings of the European Conference on Computer Vision},
 year = {2018},
} 

@inproceedings{Kaiser:2017:PCTalk,
 author = {Adrien Kaiser, Alonso Ybanez and Tamy Boubekeur},
 title = {Proxy Clouds for RGB-D Stream Processing},
 booktitle = {ACM SIGGRAPH 2017 Talk Program},
 year = {2017},
}