Estimation of flat object deformation using rgb-d sensor for robot reproduction

Xin He*, Takafumi Matsumaru

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)


This paper introduces a system that can estimate the deformation process of a deformed flat object (folded plane) and generate the input data for a robot with human-like dexterous hands and fingers to reproduce the same deformation of another similar object. The system is based on processing RGB data and depth data with three core techniques: a weighted graph clustering method for non-rigid point matching and clustering; a refined region growing method for plane detection on depth data based on an offset error defined by ourselves; and a novel sliding checking model to check the bending line and adjacent relationship between each pair of planes. Through some evaluation experiments, we show the improvement of the core techniques to conventional studies. By applying our approach to different deformed papers, the performance of the entire system is confirmed to have around 1.59 degrees of average angular error, which is similar to the smallest angular discrimination of human eyes. As a result, for the deformation of the flat object caused by folding, if our system can get at least one feature point cluster on each plane, it can get spatial information of each bending line and each plane with acceptable accuracy. The subject of this paper is a folded plane, but we will develop it into a robotic reproduction of general object deformation.

Original languageEnglish
Article number105
Pages (from-to)1-36
Number of pages36
JournalSensors (Switzerland)
Issue number1
Publication statusPublished - 2021 Jan 1


  • Bending line and folding angle
  • Deformation reproducing
  • Deformation understanding
  • Depth camera
  • Non-rigid point matching
  • Overgrowing avoiding
  • Plane detection
  • Region growing
  • SIFT de-scriptor
  • Sliding checking model
  • Weighted graph clustering

ASJC Scopus subject areas

  • Analytical Chemistry
  • Information Systems
  • Atomic and Molecular Physics, and Optics
  • Biochemistry
  • Instrumentation
  • Electrical and Electronic Engineering


Dive into the research topics of 'Estimation of flat object deformation using rgb-d sensor for robot reproduction'. Together they form a unique fingerprint.

Cite this