Repeatable Folding Task by Humanoid Robot Worker Using Deep Learning

Pin Chu Yang, Kazuma Sasaki, Kanata Suzuki, Kei Kase, Shigeki Sugano, Tetsuya Ogata

Research output: Contribution to journalArticlepeer-review

149 Citations (Scopus)


We propose a practical state-of-the-art method to develop a machine-learning-based humanoid robot that can work as a production line worker. The proposed approach provides an intuitive way to collect data and exhibits the following characteristics: task performing capability, task reiteration ability, generalizability, and easy applicability. The proposed approach utilizes a real-time user interface with a monitor and provides a first-person perspective using a head-mounted display. Through this interface, teleoperation is used for collecting task operating data, especially for tasks that are difficult to be applied with a conventional method. A two-phase deep learning model is also utilized in the proposed approach. A deep convolutional autoencoder extracts images features and reconstructs images, and a fully connected deep time delay neural network learns the dynamics of a robot task process from the extracted image features and motion angle signals. The 'Nextage Open' humanoid robot is used as an experimental platform to evaluate the proposed model. The object folding task utilizing with 35 trained and 5 untrained sensory motor sequences for test. Testing the trained model with online generation demonstrates a 77.8% success rate for the object folding task.

Original languageEnglish
Article number7762066
Pages (from-to)397-403
Number of pages7
JournalIEEE Robotics and Automation Letters
Issue number2
Publication statusPublished - 2017 Apr


  • Humanoid robots
  • learning and adaptive systems
  • motion control of manipulators
  • neurorobotics

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'Repeatable Folding Task by Humanoid Robot Worker Using Deep Learning'. Together they form a unique fingerprint.

Cite this