Motion switching with sensory and instruction signals by designing dynamical systems using deep neural network

Kanata Suzuki*, Hiroki Mori, Tetsuya Ogata

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

17 Citations (Scopus)


To ensure that a robot is able to accomplish an extensive range of tasks, it is necessary to achieve a flexible combination of multiple behaviors. This is because the design of task motions suited to each situation would become increasingly difficult as the number of situations and the types of tasks performed by them increase. To handle the switching and combination of multiple behaviors, we propose a method to design dynamical systems based on point attractors that accept (i) 'instruction signals' for instruction-driven switching. We incorporate the (ii) 'instruction phase' to form a point attractor and divide the target task into multiple subtasks. By forming an instruction phase that consists of point attractors, the model embeds a subtask in the form of trajectory dynamics that can be manipulated using sensory and instruction signals. Our model comprises two deep neural networks: A convolutional autoencoder and a multiple time-scale recurrent neural network. In this study, we apply the proposed method to manipulate soft materials. To evaluate our model, we design a cloth-folding task that consists of four subtasks and three patterns of instruction signals, which indicate the direction of motion. The results depict that the robot can perform the required task by combining subtasks based on sensory and instruction signals. And, our model determined the relations among these signals using its internal dynamics.

Original languageEnglish
Article number8405582
Pages (from-to)3481-3488
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number4
Publication statusPublished - 2018 Oct


  • AI-based methods
  • Deep learning in robotics and automation
  • humanoid robots

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'Motion switching with sensory and instruction signals by designing dynamical systems using deep neural network'. Together they form a unique fingerprint.

Cite this