TY - GEN
T1 - Video translation system using face tracking and lip synchronization
AU - Morishima, S.
AU - Ogata, Shin
AU - Nakamura, S.
PY - 2001/1/1
Y1 - 2001/1/1
N2 - We introduce a multi-modal English-to-Japanese and Japanese-to-English translation system that also translates the speaker's speech motion while synchronizing it to the translated speech. To retain the speaker's facial expression, we substitute only the speech organ's image with the synthesized one, which is made by a three-dimensional wire-frame model that is adaptable to any speaker. Our approach enables image synthesis and translation with an extremely small database. Also, we propose a method to track motion of the face from the video image. In this system, movement and rotation of the head is detected by template matching using a 3D personal face wire-frame model. By this technique, an automatic video translation can be achieved.
AB - We introduce a multi-modal English-to-Japanese and Japanese-to-English translation system that also translates the speaker's speech motion while synchronizing it to the translated speech. To retain the speaker's facial expression, we substitute only the speech organ's image with the synthesized one, which is made by a three-dimensional wire-frame model that is adaptable to any speaker. Our approach enables image synthesis and translation with an extremely small database. Also, we propose a method to track motion of the face from the video image. In this system, movement and rotation of the head is detected by template matching using a 3D personal face wire-frame model. By this technique, an automatic video translation can be achieved.
UR - http://www.scopus.com/inward/record.url?scp=84908286920&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84908286920&partnerID=8YFLogxK
U2 - 10.1109/ICME.2001.1237804
DO - 10.1109/ICME.2001.1237804
M3 - Conference contribution
AN - SCOPUS:84908286920
T3 - Proceedings - IEEE International Conference on Multimedia and Expo
SP - 649
EP - 652
BT - Proceedings - IEEE International Conference on Multimedia and Expo
PB - IEEE Computer Society
T2 - 2001 IEEE International Conference on Multimedia and Expo, ICME 2001
Y2 - 22 August 2001 through 25 August 2001
ER -