Flexible priors for robust dense stereo matching
Despite huge progress in stereo matching in recent decades, dealing with untextured or weakly textured surfaces with arbitrary surface slant remains a challenge. In this talk, I will discuss various ways to address this issue by adding constraints derived from appropriate assumptions about the scene geometry. I will first present a special-purpose stereo matching technique suitable for man-made scenes with several dominant planes and describe an extension of that technique that can be effective for image-based rendering applications. I will then discuss general purpose algorithms we have recently developed, where geometric priors such as local surface planarity and preference for known surface orientations can be efficiently incorporated as soft constraints.
In the second part of my talk, I will explain why I think more effective priors can be exploited if we solve stereo matching holistically with other complementary tasks such object segmentation, optical flow and motion segmentation. I will briefly describe how we have adopted such ideas first for joint stereo matching and object segmentation and also for scene flow recovery from stereoscopic video.
Sudipta Sinha is a researcher in the Interactive Media (IMG) Group at Microsoft Research Redmond. His research interests lie broadly in computer vision, robotics and computer graphics and he works on various topics related to 3d scene reconstruction from images and video, including structure from motion, SLAM, visual odometry, stereo matching, optical flow and scene flow, multi-view stereo, photometric stereo, image-based localization and place recognition. He is interested in applications ranging from 3D scanning, depth sensing, augmented reality (AR) and building computer vision-aided autonomous UAVs. Sudipta received his M.S. and Ph.D. from the University of North Carolina at Chapel Hill.
What levels of geometric priors do you need for 3D modeling?
Architectural scenes exhibit rich geometric patterns and regularities. I will present a sequence of our research that exploits geometric regularities for reconstructing high-quality 3D models from raw imaging sensor data. The classes of geometric priors range from low level geometric primitives such as lines, planes, or cuboids to high-level scene grammars constraining the reconstruction process.
While the use of geometric priors remain as an interesting research topic, their production use is still fairly limited, unlike dense reconstruction techniques without any priors. At the end of my talk, I would like to share my view of why this is the case.
Yasutaka Furukawa is Assistant Professor of Computer Science and Engineering at Washington University in St. Louis. He is also a principal research scientist at Zillow Group. Prior to WUSTL, he was a software engineer at Google and a post-doctoral research associate at University of Washington, working with Steve Seitz, Brian Curless and Rick Szeliski at Facebook. He has published widely in the area of 3-D reconstruction using vision, much of which is based on using planar and Manhattan models.