Data collection for mobile audio-visual speech recognition in various environments

Satoshi Tamura; Takumi Seko; Satoru Hayamizu

doi:10.1109/ICSDA.2014.7051434

Data collection for mobile audio-visual speech recognition in various environments

Satoshi Tamura, Takumi Seko, Satoru Hayamizu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

This paper introduces our recent activities for audio-visual speech recognition on mobile devices and data collection in various environments. Audio-visual automatic speech recognition is effective in noisy or real conditions to enhance the robustness of speech recognizer and to improve the recognition accuracy. We have developed an audio-visual speech recognition interface for mobile devices. In order to evaluate the recognizer and investigate issues related to audio-visual processing on mobile computers, we collected speech data and lip images of 16 subjects in eight conditions, where there were various audio noises and visual difficulties. Audio-only speech recognition and visual-only lipreading were then conducted. Through these experiments, we found some issues and future works not only for construction of audio-visual database but also for robust audio-visual speech recognition.

Original language	English
Title of host publication	Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation)
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781479970940
DOIs	https://doi.org/10.1109/ICSDA.2014.7051434
Publication status	Published - 2014 Feb 27
Externally published	Yes
Event	17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment, Oriental COCOSDA 2014 - Phuket, Thailand Duration: 2014 Sept 10 → 2014 Sept 12

Publication series

Name	Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation)

Other

Other	17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment, Oriental COCOSDA 2014
Country/Territory	Thailand
City	Phuket
Period	14/9/10 → 14/9/12

ASJC Scopus subject areas

Software
Computer Science Applications
Language and Linguistics
Linguistics and Language

Access to Document

10.1109/ICSDA.2014.7051434

Cite this

Tamura, S., Seko, T., & Hayamizu, S. (2014). Data collection for mobile audio-visual speech recognition in various environments. In Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation) Article 7051434 (Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation)). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICSDA.2014.7051434

Data collection for mobile audio-visual speech recognition in various environments. / Tamura, Satoshi; Seko, Takumi; Hayamizu, Satoru.
Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation). Institute of Electrical and Electronics Engineers Inc., 2014. 7051434 (Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation)).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Tamura, S, Seko, T & Hayamizu, S 2014, Data collection for mobile audio-visual speech recognition in various environments. in Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation)., 7051434, Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation), Institute of Electrical and Electronics Engineers Inc., 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment, Oriental COCOSDA 2014, Phuket, Thailand, 14/9/10. https://doi.org/10.1109/ICSDA.2014.7051434

Tamura S, Seko T, Hayamizu S. Data collection for mobile audio-visual speech recognition in various environments. In Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation). Institute of Electrical and Electronics Engineers Inc. 2014. 7051434. (Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation)). doi: 10.1109/ICSDA.2014.7051434

Tamura, Satoshi ; Seko, Takumi ; Hayamizu, Satoru. / Data collection for mobile audio-visual speech recognition in various environments. Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation). Institute of Electrical and Electronics Engineers Inc., 2014. (Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation)).

@inproceedings{99d932a689864ee78d180edd6f96faf5,

title = "Data collection for mobile audio-visual speech recognition in various environments",

abstract = "This paper introduces our recent activities for audio-visual speech recognition on mobile devices and data collection in various environments. Audio-visual automatic speech recognition is effective in noisy or real conditions to enhance the robustness of speech recognizer and to improve the recognition accuracy. We have developed an audio-visual speech recognition interface for mobile devices. In order to evaluate the recognizer and investigate issues related to audio-visual processing on mobile computers, we collected speech data and lip images of 16 subjects in eight conditions, where there were various audio noises and visual difficulties. Audio-only speech recognition and visual-only lipreading were then conducted. Through these experiments, we found some issues and future works not only for construction of audio-visual database but also for robust audio-visual speech recognition.",

author = "Satoshi Tamura and Takumi Seko and Satoru Hayamizu",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.; 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment, Oriental COCOSDA 2014 ; Conference date: 10-09-2014 Through 12-09-2014",

year = "2014",

month = feb,

day = "27",

doi = "10.1109/ICSDA.2014.7051434",

language = "English",

series = "Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation)",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation)",

}

TY - GEN

T1 - Data collection for mobile audio-visual speech recognition in various environments

AU - Tamura, Satoshi

AU - Seko, Takumi

AU - Hayamizu, Satoru

PY - 2014/2/27

Y1 - 2014/2/27

N2 - This paper introduces our recent activities for audio-visual speech recognition on mobile devices and data collection in various environments. Audio-visual automatic speech recognition is effective in noisy or real conditions to enhance the robustness of speech recognizer and to improve the recognition accuracy. We have developed an audio-visual speech recognition interface for mobile devices. In order to evaluate the recognizer and investigate issues related to audio-visual processing on mobile computers, we collected speech data and lip images of 16 subjects in eight conditions, where there were various audio noises and visual difficulties. Audio-only speech recognition and visual-only lipreading were then conducted. Through these experiments, we found some issues and future works not only for construction of audio-visual database but also for robust audio-visual speech recognition.

AB - This paper introduces our recent activities for audio-visual speech recognition on mobile devices and data collection in various environments. Audio-visual automatic speech recognition is effective in noisy or real conditions to enhance the robustness of speech recognizer and to improve the recognition accuracy. We have developed an audio-visual speech recognition interface for mobile devices. In order to evaluate the recognizer and investigate issues related to audio-visual processing on mobile computers, we collected speech data and lip images of 16 subjects in eight conditions, where there were various audio noises and visual difficulties. Audio-only speech recognition and visual-only lipreading were then conducted. Through these experiments, we found some issues and future works not only for construction of audio-visual database but also for robust audio-visual speech recognition.

UR - http://www.scopus.com/inward/record.url?scp=84949923993&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84949923993&partnerID=8YFLogxK

U2 - 10.1109/ICSDA.2014.7051434

DO - 10.1109/ICSDA.2014.7051434

M3 - Conference contribution

AN - SCOPUS:84949923993

T3 - Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation)

BT - Oriental COCOSDA 2014 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment / CASLRE (Conference on Asian Spoken Language Research and Evaluation)

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 17th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment, Oriental COCOSDA 2014

Y2 - 10 September 2014 through 12 September 2014

ER -

Data collection for mobile audio-visual speech recognition in various environments

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this