TY - JOUR
T1 - A Robotic Grasping State Perception Framework With Multi-Phase Tactile Information and Ensemble Learning
AU - Yan, Gang
AU - Schmitz, Alexander
AU - Funabashi, Satoshi
AU - Somlor, Sophon
AU - Tomo, Tito Pradhono
AU - Sugano, Shigeki
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2022/7/1
Y1 - 2022/7/1
N2 - Recently, tactile sensing has attracted increasing attention for robotic manipulation. Predicting the grasping stability before lifting objects and detecting the ongoing/onset of slip after lifting objects are two critical and widely studied tasks in robotic tactile manipulation. Previous methods focus on proposing novel neural networks (NN) architectures towards one of the above two tasks and did not consider that the two tasks are employed in two interconnected action-phases, i.e. grasping and lifting. Therefore, we firstly explore the possibility of constructing a multi-phase, multi-output framework to combine the stability prediction before lifting and the slip detection after lifting. Moreover, to improve the prediction/detection accuracy, we also proposed to explicitly ensemble different NN architectures using various methods, including attention mechanisms. Our experiments are done with 6 state-of-art NN architectures on two datasets including more than 3000 robotic grasps over 80 objects in total. Furthermore our proposals are tested in a real-time robot experiment with unseen objects. Our experimental results show that the proposed multi-phase, multi-output model exhibits more reliable and flexible performance than a single phase model. We also show that using the ensemble of different NN architectures is a viable and practical choice to boost the overall performance.
AB - Recently, tactile sensing has attracted increasing attention for robotic manipulation. Predicting the grasping stability before lifting objects and detecting the ongoing/onset of slip after lifting objects are two critical and widely studied tasks in robotic tactile manipulation. Previous methods focus on proposing novel neural networks (NN) architectures towards one of the above two tasks and did not consider that the two tasks are employed in two interconnected action-phases, i.e. grasping and lifting. Therefore, we firstly explore the possibility of constructing a multi-phase, multi-output framework to combine the stability prediction before lifting and the slip detection after lifting. Moreover, to improve the prediction/detection accuracy, we also proposed to explicitly ensemble different NN architectures using various methods, including attention mechanisms. Our experiments are done with 6 state-of-art NN architectures on two datasets including more than 3000 robotic grasps over 80 objects in total. Furthermore our proposals are tested in a real-time robot experiment with unseen objects. Our experimental results show that the proposed multi-phase, multi-output model exhibits more reliable and flexible performance than a single phase model. We also show that using the ensemble of different NN architectures is a viable and practical choice to boost the overall performance.
KW - Deep learning in grasping and manipulation
KW - grasping
KW - perception for grasping and manipulation
UR - http://www.scopus.com/inward/record.url?scp=85124842136&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85124842136&partnerID=8YFLogxK
U2 - 10.1109/LRA.2022.3151260
DO - 10.1109/LRA.2022.3151260
M3 - Article
AN - SCOPUS:85124842136
SN - 2377-3766
VL - 7
SP - 6822
EP - 6829
JO - IEEE Robotics and Automation Letters
JF - IEEE Robotics and Automation Letters
IS - 3
ER -