Visual motor integration of robot's drawing behavior using recurrent neural network

Kazuma Sasaki*, Kuniaki Noda, Tetsuya Ogata

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

25 Citations (Scopus)


Drawing is a way of visually expressing our feelings, knowledge, and situation. People draw pictures to share information with other human beings. This study investigates visuomotor memory (VM), which is a reusable memory storing drawing behavioral data. We propose a neural network-based model for acquiring a computational memory that can replicate VM through self-organized learning of a robot's actual drawing experiences. To design the model, we assume that VM has the following two characteristics: (1) it is formed by bottom-up learning and integration of temporal drawn pictures and motion data, and (2) it allows the observers to associate drawing motions from pictures. The proposed model comprises a deep neural network for dimensionally compressing temporal drawn images and a continuous-time recurrent neural network for integration learning of drawing motions and temporal drawn images. Two experiments are conducted on unicursal shape learning to investigate whether the proposed model can learn the function without any shape information for visual processing. Based on the first experiment, the model can learn 15 drawing sequences for three types of pictures, acquiring associative memory for drawing motions through the bottom-up learning process. Thus, it can associate drawing motions from untrained drawn images. In the second experiment, four types of pictures are trained, with four distorted variations per type. In this case, the model can organize the different shapes based on their distortions by utilizing both the image information and the drawing motions, even if visual characteristics are not shared.

Original languageEnglish
Pages (from-to)184-195
Number of pages12
JournalRobotics and Autonomous Systems
Publication statusPublished - 2016 Dec 1


  • Deep learning
  • Drawing ability
  • Drawing robot

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Mathematics(all)
  • Computer Science Applications


Dive into the research topics of 'Visual motor integration of robot's drawing behavior using recurrent neural network'. Together they form a unique fingerprint.

Cite this