Automatic estimation of the position and orientation of stairs to be reached and climbed by a disaster response robot by analyzing 2D image and 3D point cloud

In order to realize a disaster response robot that can reach and climb straight stairs within a certain range, this paper proposes a method for estimating the position and orientation of the stairs using 2D image and 3D point cloud. In this method, first, an object detection method is applied to an RGB image, and a 3D point cloud including stairs is extracted by combining the detection result and the 3D point cloud. Next, a 3D point cloud of a step candidate is extracted by applying plane estimation and region segmentation to the extracted 3D point cloud. The 3D point cloud of the step candidate is projected on a 2D plane, and the orientation of the stairs is estimated by detecting their contour and lines. In addition, the position of the stairs is estimated by searching for a combination of 3D point clouds of the step candidates located at equal intervals using the structural characteristics of the stairs. As a result of simulation using a disaster response robot WAREC-1, it was confirmed that the orientation of the stairs can be accurately estimated by the proposed method. It was also confirmed that the position could be accurately estimated under specific conditions.