BIB-VERSION:: CS-TR-v2.0
ID:: ncstrl.dartmouthcs//TR2004-501
ENTRY:: June 04, 2004
ORGANIZATION:: Dartmouth College, Computer Science
TITLE:: Synchronizing Keyframe Facial Animation to Multiple Text-to-Speech Engines and  Natural Voice with Fast Response Time
TYPE:: Technical Report (paper)
REVISION:: 1
AUTHOR:: Pechter, William
DATE:: June 2004
RETRIEVAL:: For a paper copy, email <reports@cs.dartmouth.edu>
RETRIEVAL:: For a paper copy, write to
       Technical Report Librarian
       Department of Computer Science
       Dartmouth College
       6211 Sudikoff Laboratory
       Hanover, NH 03755-3510
       USA
RETRIEVAL:: PDF at http://www.cs.dartmouth.edu/reports/TR2004-501.pdf
ABSTRACT:: 
This thesis aims to create an automated lip-synchronization system for real-time applications.  Specifically, the system is required to be fast, consist of a limited number of keyframes with small memory requirements, and create fluid and believable animations that synchronize with text-to-speech engines as well as raw voice data. 
  The algorithms utilize traditional keyframe animation and a novel method of keyframe selection.  Additionally, phoneme-to-keyframe mapping, synchronization, and simple blending rules are employed.  The algorithms provide blending between keyframe images, borrow information from neighboring phonemes, accentuate phonemes b, p and m, differentiate between keyframes for phonemes with allophonic variations, and provide prosodromic variation by including emotion while speaking.  The lip-sync animation synchronizes with multiple synthesized voices and human speech.  A fast and versatile online real-time java chat interface is created to exhibit vivid facial animation.
  Results show that the animation algorithms are fast and show accurate lip-synchronization.  Additionally, surveys showed that the animations are visually pleasing and improve speech understandability 96% of the time.  Applications for this project include internet chat capabilities, interactive teaching of foreign languages, animated news broadcasting, enhanced game technology, and cell phone messaging.
NOTE:: 
Senior Honors Thesis.  Advisors: Lorie Loeb, Hany Farid, and Stephen Linder
END:: ncstrl.dartmouthcs//TR2004-501