Conversion of sensitivity-based tasks from brain signals and motions: Applications to humanoid operation

Multimodal signals emanated from human users are applied to operations of bipedal humanoids. Distinctive features of the designed system include recognition and conversion of sensibilities as patterns contained in the biosignals. The total recognition system is a combination of Bayesian networks, hidden Markov models, independent component analysis, and support vector machines. Input biosignals are electro-encephalogram, brain near infra-red spectroscopy, neural spike trains, and body motions (gestures). After the recognition of biosignals, user intentions issued as patterns are transduced to different electronic tasks. In addition to such an ability of sensory conversion, this mechanism has a merit of enhancing the independence of target machines. The combined recognizer allows system designers to use conventional PCs alone. With all of such advantages, a bipedal humanoid, which is selected as a representative of contemporary electronic appliances, is operated by the human biosignals. Operations by inner language patterns are also realized.