Achieving Human–Robot Collaboration with Dynamic Goal Inference by Gradient Descent

Shingo Murata*, Wataru Masuda, Jiayi Chen, Hiroaki Arie, Tetsuya Ogata, Shigeki Sugano


研究成果: Conference contribution

7 被引用数 (Scopus)


Collaboration with a human partner is a challenging task expected of intelligent robots. To realize this, robots need the ability to share a particular goal with a human and dynamically infer whether the goal state is changed by the human. In this paper, we propose a neural network-based computational framework with a gradient-based optimization of the goal state that enables robots to achieve this ability. The proposed framework consists of convolutional variational autoencoders (ConvVAEs) and a recurrent neural network (RNN) with a long short-term memory (LSTM) architecture that learns to map a given goal image for collaboration to visuomotor predictions. More specifically, visual and goal feature states are first extracted by the encoder of the respective ConvVAEs. Visual feature and motor predictions are then generated by the LSTM based on their current state and are conditioned according to the extracted goal feature state. During collaboration after the learning process, the goal feature state is optimized by gradient descent to minimize errors between the predicted and actual visual feature states. This enables the robot to dynamically infer situational (goal) changes of the human partner from visual observations alone. The proposed framework is evaluated by conducting experiments on a human–robot collaboration task involving object assembly. Experimental results demonstrate that a robot equipped with the proposed framework can collaborate with a human partner through dynamic goal inference even when the situation is ambiguous.

ホスト出版物のタイトルNeural Information Processing - 26th International Conference, ICONIP 2019, Proceedings
編集者Tom Gedeon, Kok Wai Wong, Minho Lee
出版ステータスPublished - 2019
イベント26th International Conference on Neural Information Processing, ICONIP 2019 - Sydney, Australia
継続期間: 2019 12月 122019 12月 15


名前Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
11954 LNCS


Conference26th International Conference on Neural Information Processing, ICONIP 2019

ASJC Scopus subject areas

  • 理論的コンピュータサイエンス
  • コンピュータサイエンス一般


「Achieving Human–Robot Collaboration with Dynamic Goal Inference by Gradient Descent」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。