MODELING TONES IN CONTINUOUS CANTONESE SPEECH Tan LEE

MODELING TONES IN CONTINUOUS CANTONESE SPEECH Tan LEE*, Greg KOCHANSKI†, Chilin SHIH† and Yujia LI* *Department of Electronic Engineering, the Chinese University of Hong Kong, Shatin, N. T. , Hong Kong †Bell Laboratories, Lucent Technologies, 600 Mountain Avenue, Murray Hill, NJ 07974, U. S. A. Acknowledgement This research is substantially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region (Project No. CUHK 4291/00 E). STEM-ML: Soft TEMplate Mark-up Language Experimental Results Quantitative modeling of Cantonese tones and intonation based on the STEM -ML framework by Kochanski and Shih, Prosody modelling with soft templates, to appear in Speech Communication, 2002. http: //prosodies. org/bell/ Speech Data • How the prosodic strength of a tone is related to word position, word length, sentence position, part-of-speech, etc. ? • How to describe the phrase level intonation of Cantonese ? STEM-ML involves: a set of linguistically inspired tags, and an underlying mathematical model for F 0 prediction • Does Cantonese have 9 tones or 6 tones ? Highlights • Each tag has several adjustable parameters that control the shape of accent templates, the way they interact, or characteristics of the speaker. Cantonese • • F 0 One of the major Chinese dialects Monosyllabic and tonal Syllable = Initial + Final + Tone 19 Initials, 53 Finals, 9 Tones F 0 Profile (Tone) Initial Final (vowel nucleus + optional consonant coda) Tones in Cantonese F 0 2 3 4 5 6 7(1) 8(3) 9(6) Entering tones are seen as truncated counterparts of nonentering tones Acoustic realization of tones in isolation Tone 1, 3, 4 and 6 have flat or slightly falling F 0 movement, while Tone 2 and 5 are rising. Tone modeling A parameter, referred to as prosodic strength, controls the degree of allowed modification. This is related to how carefully the tone needs to be articulated. A “strong” tone tends to follow the original template accurately, while a “weak” tone will compromise with the context. F 0 contour is generated to balance between the effort required for articulation and the accuracy that the tone is communicated. Design of Cantonese Tone Models To maintain tone identity, the contour’s height is more crucial than its shape. Tone templates Phrase curve (Dashed line illustrates entering tones) Five-point for non-entering tones Three-point for entering tones Each intonation phrase is a straight line F 0 (t 2, p 2) Digit sequence: 4 3 9 6 9 7 0 5 2 1 4 7 5 8 Tone sequence: 3 1 2 6 2 1 4 5 6 1 3 1 5 3 100 * Phrases are defined as those separated by a perceivable pause ** Only voiced frames with valid F 0 values are used. F 0 T 6 — model with 6 -tone classification T 9 — model with 9 -tone classification Tone shapes time in seconds Prosodic Strength Tone F 0 normalized time Phrase curves • 85% of the phrases show declining intonation • overall average slope: -10. 9 Hz / second Word strength & tone strength Tones in continuous speech F 0 / Hz 200 RMS Errors T 6 T 9 11. 7 Hz 11. 0 Hz 10. 5 Hz 9. 3 Hz 11. 9 Hz N/A normalized time (t 1, p 1) time 300 Set A: Set B: Set C: Tone (or other type of local accent) is represented as a soft template. Non-entering tones (6 -tone system) Fitting accuracy Given a speech corpus, prosody modeling is done by determining optimal parameter values that give the best fit of F 0 for all utterances in the corpus. (Consonant) 1 The parameter values and the mathematical model together generate F 0 contours deterministically. • Female speaker • 30 continuous utterances divided into Set A, B and C • Text processing: manually marked word boundaries; manually labeled POS • Acoustic processing: F 0 extraction; phrase boundaries detection; voiced/unvoiced decision Surface F 0 contour is determined by - Necessity of maintaining tone identities - Physiological constraint: smoothness in F 0 movement - Co-articulation of neighboring tones - Focus condition of a word - Phrase intonation - Tone is carried by syllable. Several syllables form a word - Each word occurrence is assigned a strength parameter - Syllables in a word follows a metrical pattern that depends only on word length bui 3_ji 5 Word strength: sat 9_jim 6_sat 7 2. 0 2. 5 [0. 5 0. 3] [0. 4 0. 2] Tone strength: bui 3 ji 5 sat 9 2. 5 2. 3 2. 9 jim 6 sat 7 2. 9 2. 7 metrical pattern for 2 -syllable word metrical pattern for 3 -syllable word time in seconds Conclusions - STEM-ML is effective in modeling Cantonese prosody - Cantonese shows very similar word strength pattern and word metrical pattern with to Mandarin. - Cantonese exhibits a consistent phrasal F 0 downtrend - There is no obvious advantage of separating entering tones from non-entering tones.