A multimodal human-machine interface enabling situation-Adaptive control inputs for highly automated vehicles

Udara E. Manawadu, Mitsuhiro Kamezaki, Masaaki Ishikawa, Takahiro Kawano, Shigeki Sugano

Research output: Chapter in Book/Report/Conference proceedingConference contribution

22 Citations (Scopus)


Intelligent vehicles operating in different levels of automation require the driver to fully or partially conduct the dynamic driving task (DDT) and to conduct fallback performance of the DDT, during a trip. Such vehicles create the need for novel human-machine interfaces (HMIs) designed to conduct high-level vehicle control tasks. Multimodal interfaces (MMIs) have advantages such as improved recognition, faster interaction, and situation-Adaptability, over unimodal interfaces. In this study, we developed and evaluated a MMI system with three input modalities; touchscreen, hand-gesture, and haptic to input tactical-level control commands (e.g. lane-changing, overtaking, and parking). We conducted driving experiments in a driving simulator to evaluate the effectiveness of the MMI system. The results show that multimodal HMI significantly reduced the driver workload, improved the efficiency of interaction, and minimized input errors compared with unimodal interfaces. Moreover, we discovered relationships between input types and modalities: location-based inputs-Touchscreen interface, time-critical inputs-haptic interface. The results proved the functional advantages and effectiveness of multimodal interface system over its unimodal components for conducting tactical-level driving tasks.

Original languageEnglish
Title of host publicationIV 2017 - 28th IEEE Intelligent Vehicles Symposium
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781509048045
Publication statusPublished - 2017 Jul 28
Event28th IEEE Intelligent Vehicles Symposium, IV 2017 - Redondo Beach, United States
Duration: 2017 Jun 112017 Jun 14

Publication series

NameIEEE Intelligent Vehicles Symposium, Proceedings


Other28th IEEE Intelligent Vehicles Symposium, IV 2017
Country/TerritoryUnited States
CityRedondo Beach

ASJC Scopus subject areas

  • Computer Science Applications
  • Automotive Engineering
  • Modelling and Simulation


Dive into the research topics of 'A multimodal human-machine interface enabling situation-Adaptive control inputs for highly automated vehicles'. Together they form a unique fingerprint.

Cite this