We present a structure-from-motion framework that detects and conforms to structural regularities, while simultaneously recovering 3D geometry starting from a set of facade images. A novel graph based global analysis yields a globally consistent 3D geometry reconstruction with explicit encoding of the facade regularities.

related projects:

Coupled Structure-from-Motion and 3D Symmetry Detection for Urban Facades

ACM Transactions on Graphics, 2014

Abstract

Repeated structures are ubiquitous in urban facades. Such repetitions lead to ambiguity in establishing correspondences across sets of unordered images. A decoupled structure-from-motion reconstruction followed by symmetry detection often produces errors: outputs are either noisy and incomplete, or even worse, appear to be valid but actually have a wrong number of repeated elements. We present an optimization framework for extracting repeated elements in images of urban facades, while simultaneously calibrating the input images and recovering the 3D scene geometry using a graph-based global analysis. We evaluate the robustness of the proposed scheme on a range of challenging examples containing widespread repetitions and non-distinctive features. These image sets are common but cannot be handled well with state-of-the-art methods. We show that the recovered symmetry information along with the 3D geometry enables a range of novel image editing operations that maintain consistency across the images.


Results


Starting from a set of input images and a sample repeated element(s) marked on a single image (shown in orange), we simultaneously recover the 3D repetition patterns, calibrate the cameras, and reconstruct the scene geometry. The information can be used for a range of image manipulations.




Our approach can successfully handle buildings with multiple facades and recover the 3D repetition pattern on each facade.