Multi-scale feature selection in stereo

In binocular stereo matching, points in left and right images are matched according to features that characterize each point and identify pairs of points. When one tries to use multiple features, a difficult problem is which feature, or combination of features, to use. Moreover, features are difficult to cross-normalize and so comparisons must take into account not only their output, but also their distribution (their output for different parameters). We present a new approach that uses geometric constraints on the matching surface to select optimal feature or combination of features from multiscale-edge and intensity features. The approach requires the cyclopean coordinate system to set mutually exclusive matching choices. To obtain the matching surface, we solve a global optimization problem on an energy functional that models occlusions, discontinuities, and inter-epipolar-line interactions.