TY - GEN
T1 - Detection of Slip from Vision and Touch
AU - Yan, Gang
AU - Schmitz, Alexander
AU - Tomo, Tito Pradhono
AU - Somlor, Sophon
AU - Funabashi, Satoshi
AU - Sugano, Shigeki
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Detecting the onset/ongoing of slip, i.e. if a grasped object is slipping or will slip from the gripper while being lifted, is crucial. Conventionally, it is regarded as a tactile sensing related problem. However, recently multi-modal robotic learning has become popular and is expected to boost the performance. In this paper we propose a novel CNN-TCN model to fuse tactile and visual information for detecting the onset/ongoing of slip. In our experiments, two uSkin tactile sensors and one Realsense435i camera are used. Data is collected by randomly grasping and lifting 35 daily objects 1050 times in total. Furthermore, we compare our CNN-TCN model with the widely used CNN-LSTM model. As a result, our proposed model achieves a 88.75% detection accuracy and outperforms the CNN-LSTM model combined with different pretrained vision networks.
AB - Detecting the onset/ongoing of slip, i.e. if a grasped object is slipping or will slip from the gripper while being lifted, is crucial. Conventionally, it is regarded as a tactile sensing related problem. However, recently multi-modal robotic learning has become popular and is expected to boost the performance. In this paper we propose a novel CNN-TCN model to fuse tactile and visual information for detecting the onset/ongoing of slip. In our experiments, two uSkin tactile sensors and one Realsense435i camera are used. Data is collected by randomly grasping and lifting 35 daily objects 1050 times in total. Furthermore, we compare our CNN-TCN model with the widely used CNN-LSTM model. As a result, our proposed model achieves a 88.75% detection accuracy and outperforms the CNN-LSTM model combined with different pretrained vision networks.
UR - http://www.scopus.com/inward/record.url?scp=85136333089&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85136333089&partnerID=8YFLogxK
U2 - 10.1109/ICRA46639.2022.9811589
DO - 10.1109/ICRA46639.2022.9811589
M3 - Conference contribution
AN - SCOPUS:85136333089
T3 - Proceedings - IEEE International Conference on Robotics and Automation
SP - 3537
EP - 3543
BT - 2022 IEEE International Conference on Robotics and Automation, ICRA 2022
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 39th IEEE International Conference on Robotics and Automation, ICRA 2022
Y2 - 23 May 2022 through 27 May 2022
ER -