Palpation Robot System - Reproduction Method by Deep Neural Network of Skin Palpation Judgment Focusing on Softness Classification

In recent years, the spread of infectious diseases, such as COVID-19, has increased the need for medical examinations to avoid contact between doctors and patients. Most treatments, especially dermatology, require palpation, and its impact is significant. In this study, we aimed to reproduce the judgment of the softness and surface textures of diseased parts, which is important to dermatologists for determining the condition, using a simple robot device. Five levels of softness and three types of surface textures labeled with 14 types of materials were obtained from interviews with dermatologists. To acquire a haptic response from materials during pushing, 1) a single-rod probe with a haptic sensor using a linear actuator and 2) a dual-rod type configuration to obtain vibration propagation was constructed. Frequency-analyzed images were produced from the obtained waveforms of force and acceleration. A total of 343 images from 13 materials were used for transfer learning and were classified using AlexNet. The classification accuracy of the single-rod probe was 93.0%, and that of the dual-probe configuration was 95.2%. The classification accuracy was improved using the dual probe configuration than the single one; the softness classification accuracy was improved from 93.8% (single-rod) to 95.7% (dual-rod configuration). The surface texture classification accuracy was improved from 91.9% (single-rod) to 92.8% (dual-rod configuration), respectively. Therefore, the proposed method enables the reproduction of the judgment of five-level softness and three types of surface texture judgment by dermatologists.