ECA control using a single affective User dimension

Fred Charles, Florian Pecune, Gabor Aranyi, Catherine Pelachaud, Marc Cavazza*


研究成果: Conference contribution

3 被引用数 (Scopus)


User interaction with Embodied Conversational Agents (ECA) should involve a significant affective component to achieve realism in communication. This aspect has been studied through different frameworks describing the relationship between user and ECA, for instance alignment, rapport and empathy. We conducted an experiment to explore how an ECA's non-verbal expression can be controlled to respond to a single affective dimension generated by users as input. Our system is based on the mapping of a high-level affective dimension, approach/avoidance, onto a new ECA control mechanism in which Action Units (AU) are activated through a neural network. Since 'approach' has been associated to prefrontal cortex activation, we use a measure of prefrontal cortex left-asymmetry through fNIRS as a single input signal representing the user's attitude towards the ECA. We carried out the experiment with 1 0 subjects, who have been instructed to express a positive mental attitude towards the ECA. In return, the ECA facial expression would reflect the perceived attitude under a neurofeedback paradigm. Our results suggest that users are able to successfully interact with the ECA and perceive its response as consistent and realistic, both in terms of ECA responsiveness and in terms of relevance of facial expressions. From a system perspective, the empirical calibration of the network supports a progressive recruitment of various AUs, which provides a principled description of the ECA response and its intensity. Our findings suggest that complex ECA facial expressions can be successfully aligned with one high-level affective dimension. Furthermore, this use of a single dimension as input could support experiments in the fine-tuning of AU activation or their personalization to user preferred modalities.

ホスト出版物のタイトルICMI 2015 - Proceedings of the 2015 ACM International Conference on Multimodal Interaction
出版社Association for Computing Machinery, Inc
出版ステータスPublished - 2015 11月 9
イベントACM International Conference on Multimodal Interaction, ICMI 2015 - Seattle, United States
継続期間: 2015 11月 92015 11月 13


名前ICMI 2015 - Proceedings of the 2015 ACM International Conference on Multimodal Interaction


ConferenceACM International Conference on Multimodal Interaction, ICMI 2015
国/地域United States

ASJC Scopus subject areas

  • コンピュータ サイエンスの応用
  • コンピュータ ビジョンおよびパターン認識
  • ハードウェアとアーキテクチャ
  • 人間とコンピュータの相互作用


「ECA control using a single affective User dimension」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。