3D reconstruction of outdoor scenes using structure from motion and depth data

Abstract

Recently, low-cost and small RGB-D sensors appear massively at the entertainment market. These sensors can acquire colored 3D models using color images and depth data. However, a limitation of the RGB-D sensor is that sunlight interferes with the pattern projecting LED. The sensor is most suitable only for indoor scenes. Some RGBD sensors are available in outdoor scenes. However, the measurement range is limited because the light of LED spreads in all directions. In this research, we developed a novel measurement method for RGB-D sensors, which can measure shapes in outdoor scenes. This method uses several measurement data from multiple viewpoints, and estimates the shape and the sensor poses using Structure from Motion (SfM). However, a conventional imagebased SfM cannot determine a correct scale. To determine the correct scale, our method uses the depth information that is obtained from partially acquired area which is near to the viewpoints. Then, our method optimizes the shape and the poses by a modified bundle adjustment with the depth information. It minimizes the reprojection error of the features in the acquired images and the depth error between the estimated model and the measurement depth. At last, our method generates dense point cloud using a multi-view stereo algorithm. Using both the acquired images and depth data, our method reconstructs the shape which locates out of measurement range in outdoor environment. In our experiment, we show that our method can measure the range up to 20 meters away by measuring from several viewpoints in the range of 5 meters using a RGB-D sensor in outdoor scenes.