Towards Synthesis of Focus in Mandarin Texttospeech System

Towards Synthesis of Focus in Mandarin Text-to-speech System Dr. Dezhi HUANG dezhi. huang@francetelecom. cn SNLP Unit, FTRD Beijing 2005/11/2 V 1. 1

Table of Contents 1 Synthesis of focus 2 Proposal for SSML 3 Examples with <focus> 2

Human has the strong ability of information reconstruct èEvidence from music perception The “Butterfly Lovers” violin concerto 3

Human has the strong ability of information reconstruct (Cont. ) èEvidence from human vision 4

Application model of Mandarin Text-to-speech (Cont. ) èSpoken dialog system Information query by the side of road Mandarin Voice-enabled Service Gateway PSTN/ Wireless Mandarin TTS Engine Angry Environment Noise 5

Why we fail? èThe important content is not prominent as we expect l Weaken the background noise (Noise reduction) l Improve the prominence of information that we need l Utilizing the human ability of information reconstruct 6

What do we need in speech communication? èThe key information is always contained in a phrase/word in a sentence l Have you always seen Prof. Zhao? No, I saw him only once. l The container of key information is called the focus. l l The semantic centre of a sentence 7

The value of synthesis of focus èIt is helpful for l Analyzing the syntactic of sentence l Understanding the meaning of utterance l Capturing the turn-taking l Comprehending the attempt and emotion of speaker èImprove the acceptance of TTS 8

Key challenges in synthesis of focus è Difficult to locate a focus in a sentence l Some focuses can be found from the syntactic structure l l 明天你准备去买什么？我要去买红色的帽子。 The other focuses are decided by the context of a sentence l l l 老王去年退休了。 Markup Language for Focus è Lack of appropriate acoustic model to realize a focus l l l Pitch accent Duration Energy Pause Weakness 9

Table of Contents 1 Synthesis of focus Make the synthesized speech clear Improve the validity of speech communication with TTS 2 Proposal for SSML 3 Examples with <focus> 10

What is SSML? èIt is designed to provide a rich, XML-based markup language for assisting the generation of synthetic speech in Web and other applications SSML Natural Language Processing and Understanding Speech Synthesis 11

<EMPHASIS> in SSML èThe emphasis element requests that the contained text be spoken with emphasis (also referred to as prominence or stress) l Level: strong, moderate and none l For synthesizer, it is easy to know which word has sentence stress l l 老王买了车。 12

The proposed <focus> element èThe focus element indicates that the contained text be the semantic centre and the carrier of important information of a sentence èIn the perspective of pragmatics l Contrastive focus (also referred to as identificational focus) l Informational focus (also referred to as the presentational focus, natural focus) 13

A focus in Mandarin is not one-to-one corresponding with an emphasis èMost of focuses are realized by stresses l 是老张退休了。 l 明天最高气温多少度？明天最高气温 30度。 èSome of them are realized by pause or intonation l 你常常见赵老师吗？我见过他一次。 l 我们下象棋吧。 15

Differences between focus and emphasis èFocus is the concept of semantics and pragmatics l l We can mark the focus up without speech signal 国家 商总局昨天发出紧急通知强调，全国大中城市、边境地区、发 生过疫情的地区、养殖大省四类区域必须建立健全禽类产品“挂牌经 营”制度，市场内禽类产品要标明禽类生产地、动物检验检疫证明及 销售承诺。 èEmphasis is the concept of psychoacoustics l The consistency of emphasis label is relatively difficult to achieve without speech signal 16

Differences between focus and emphasis (Cont. ) èFocus always carries the purpose of utterance l We can know exactly what the sentence means èEmphasis is not directly linked to the purpose of utterance l The emphasized word may be trivial l 黄菊强调，认真学习贯彻五中全会精神，继续推进国有商 业银行改革。 l 他经常和我打球。 17

What can we benefit from focus labeling? èImprove the intelligibility of synthesized speech, especially in communication environment with noise Q: 明天最晚一班到北京的飞机是几点？ A: 在晚上9点钟有一班CZ 8071的飞机飞往北京。 Q: 几点钟？ A: 是 9点。 Q: 哪一班？ A: 是CZ 8071。 18

What can we benefit from focus labeling? (Cont. ) èfocus labeling can be directly applied to text information processing l The next generation of search engine should need to know l l which is the topic of a paragraph which are the focuses of a sentence l Text highlight is important step for information retrieval l Keywords in automatic digest are always the focuses 19

Table of Contents 1 Synthesis of focus 2 Proposal for SSML <focus> indicates what is semantic centre 3 <focus> solves the problem of focus location Examples with <focus> 20

Attributes of <focus> èType l informational l contrastive èMethod l l l Strong. Stress Moderate. Stress None Pause Intonation 21

Samples of <focus> è (1) 你经常见<focus type=“informational” method=“Strong. Stress ”>赵教授</focus>吗？ è 我见过他<focus type=“informational” method=“Pause”> 一次</focus>。 è (2)昨天老张干什么了？ è 昨天老张<focus type=“informational” method=“Moderate. Stress ”>去看病</focus>。 è (3)是<focus type=“contrastive” method=“Strong. Stress ”>老 张</focus>帮我修了车。 22

Samples of <focus> (Cont. ) è (4)他连<focus type=“contrastive” method=“Strong. Stress ”> 我</focus>也不相信。 è (5)他经常<focus type=“informational” method=“Pause”>和 我打球</focus>。 è (6)他居然<focus type=“informational” method=“Moderate. Stress ”>卖了房子</focus>。 è (7)我们<focus type=“informational” method=“Intonation ”>去 钓鱼</focus>吧。 23

Thank you! 24