Speech Synthesis Markup Language Aim at Extension Dr

  • Slides: 19
Download presentation
Speech Synthesis Markup Language -----Aim at Extension Dr. Jianhua Tao National Laboratory of Pattern

Speech Synthesis Markup Language -----Aim at Extension Dr. Jianhua Tao National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academic of Sciences

Brief Introduction to Evolution of SSML o o o The original SSML (not W

Brief Introduction to Evolution of SSML o o o The original SSML (not W 3 C SSML) STML JSML SABLE W 3 C SSML … National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academic of Sciences

The original SSML o o Mark phrase boundaries Emphasis words Specify pronunciations Include other

The original SSML o o Mark phrase boundaries Emphasis words Specify pronunciations Include other sound files National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academic of Sciences

STML o o o Developed by Edinburgh and Bell Labs Based on the original

STML o o o Developed by Edinburgh and Bell Labs Based on the original SSML Aimed at giving the same basic impressions to listeners, not sounding identical on different systems National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academic of Sciences

JSML o o o Developed by Sun XML based Include n n n Elements

JSML o o o Developed by Sun XML based Include n n n Elements to mark the paragraphs and sentences Elements to control the pronunciations Elements to represent markers National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academic of Sciences

SABLE o o o Developed by Edinburgh and Bell Labs Based on STML and

SABLE o o o Developed by Edinburgh and Bell Labs Based on STML and JSML The stated aims n Synthesizer control o o n n Text structure Speech pronunciation Multilinguality Easy of Use Portable Extensibility National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academic of Sciences

W 3 C SSML o Key design criteria n n n Consistency Interoperability Generality

W 3 C SSML o Key design criteria n n n Consistency Interoperability Generality Internationalization Generation and Readability Implementable National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academic of Sciences

What we want from markup language o o o Controlling Sharing Extended to multimedia

What we want from markup language o o o Controlling Sharing Extended to multimedia National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academic of Sciences

Which level we should focus o o o Text analysis module Prosody module Acoustic

Which level we should focus o o o Text analysis module Prosody module Acoustic module

Data Structure 1 Data Structure 2 Sharing Sys 2 Text-analysis Prosody-analysis acoustic SSML Text-analysis

Data Structure 1 Data Structure 2 Sharing Sys 2 Text-analysis Prosody-analysis acoustic SSML Text-analysis SSML Sys 1 Prosody-analysis acoustic National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academic of Sciences

Text level for Mandarin o o Word boundary Pronunciation with tone POS Dialect?

Text level for Mandarin o o Word boundary Pronunciation with tone POS Dialect?

Prosody level for Mandarin o o Tone sandhi Rhythm ?

Prosody level for Mandarin o o Tone sandhi Rhythm ?

Extensions to expressive synthesis o o Emotion and Style Others National Laboratory of Pattern

Extensions to expressive synthesis o o Emotion and Style Others National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academic of Sciences

Current elements related to prosody and style in SSML o o 3. 2. 1

Current elements related to prosody and style in SSML o o 3. 2. 1 3. 2. 2 3. 2. 3 3. 2. 4 "voice" Element "emphasis" Element "break" Element "prosody" Element

Emotion and Style o Emotion n o Anger, happy, surprise, sad, fear, … Depend

Emotion and Style o Emotion n o Anger, happy, surprise, sad, fear, … Depend on speaker’s psychological and physical states Local effects on prosody Style n n n News, comments, … Depend on semantics of sentences Global effects on prosody

Personalized Voice o Element：voice n n n o o “gender”： “age”： “name”： “variant”： sample：

Personalized Voice o Element：voice n n n o o “gender”： “age”： “name”： “variant”： sample： 他说：<voice gender=”male”>“什么意思？ ”</voice> 她回答：<voice gender=”female”>“没什么 意思。”</voice>

Extension? o To make it more expressive n n Background music VTTS o n

Extension? o To make it more expressive n n Background music VTTS o n n Combined with talking head and some other media information … We only can see the element “mark“ National Laboratory of Pattern Recognition (NLPR) Institute of Automation, Chinese Academic of Sciences

Thanks!

Thanks!

o Element: <Structure> n n o Level: 0 -. . ; paragraph, phrase, POS:

o Element: <Structure> n n o Level: 0 -. . ; paragraph, phrase, POS: <Structure: level=paragraph> n <Structure: level=sentence> o <Structure: level=phrase> n <Structure: level=word>

Speech synthesis 1 Speech synthesis What is theSpeech synthesis 1 Speech synthesis What is the
Reported speech Indirect speech Direct speech Quoted speechReported speech Indirect speech Direct speech Quoted speech
Reported speech Indirect speech Direct speech Quoted speechReported speech Indirect speech Direct speech Quoted speech
HTML Hypertext Markup Language XML e Xtensible MarkupHTML Hypertext Markup Language XML e Xtensible Markup
Pertemuan 7 Wireless Markup Language WML Wireless MarkupPertemuan 7 Wireless Markup Language WML Wireless Markup
XML Extensible Markup Language 1 HTML Hypertext MarkupXML Extensible Markup Language 1 HTML Hypertext Markup
Speech Language Speech and Language explained Communication isSpeech Language Speech and Language explained Communication is
Speech and Language Therapy Developing Speech Language andSpeech and Language Therapy Developing Speech Language and
OCDSB Speech Language Pathology Services Speech Language PathologistsOCDSB Speech Language Pathology Services Speech Language Pathologists
speech and language disorders Speech Language disorders 1speech and language disorders Speech Language disorders 1
Introduction to Speech Language Disorder 1 Speech LanguageIntroduction to Speech Language Disorder 1 Speech Language
PROTEIN SYNTHESIS Jessica Hawley PROTEIN SYNTHESIS Protein SynthesisPROTEIN SYNTHESIS Jessica Hawley PROTEIN SYNTHESIS Protein Synthesis
PROTEIN SYNTHESIS Protein Synthesis The production synthesis ofPROTEIN SYNTHESIS Protein Synthesis The production synthesis of
PROTEIN SYNTHESIS 1 Protein Synthesis The production synthesisPROTEIN SYNTHESIS 1 Protein Synthesis The production synthesis
PROTEIN SYNTHESIS 1 Protein Synthesis The production synthesisPROTEIN SYNTHESIS 1 Protein Synthesis The production synthesis
MARKUP MARKUP n The difference in the amountMARKUP MARKUP n The difference in the amount
Markup and Markdown Find the cost markup orMarkup and Markdown Find the cost markup or
Application of Speech Recognition Synthesis Dialog Speech forApplication of Speech Recognition Synthesis Dialog Speech for
Speech Processing Analysis and Synthesis of PoleZero SpeechSpeech Processing Analysis and Synthesis of PoleZero Speech
5 Text To Speech TTS Speech Synthesis Concept5 Text To Speech TTS Speech Synthesis Concept
Speech Recognition Speech Signal Representations Veton Kpuska SpeechSpeech Recognition Speech Signal Representations Veton Kpuska Speech
Tutorial Working with Speech Speech Input Form SpeechTutorial Working with Speech Speech Input Form Speech
8 Speech Recognition Speech Recognition Concepts Speech Recognition8 Speech Recognition Speech Recognition Concepts Speech Recognition
IMPROMPTU SPEECH Tips on Impromptu Speech IMPROPTU SPEECHIMPROMPTU SPEECH Tips on Impromptu Speech IMPROPTU SPEECH
Types of Speech Pure Speech Symbolic Speech PureTypes of Speech Pure Speech Symbolic Speech Pure
Speech Processing Speech Coding Veton Kpuska Speech CodingSpeech Processing Speech Coding Veton Kpuska Speech Coding
Direct speech and reported speech Direct speech LookDirect speech and reported speech Direct speech Look
Reported Speech Direct Speech Reported Speech Present SimpleReported Speech Direct Speech Reported Speech Present Simple
Reported Speech vs Direct Speech Reported speech summarisesReported Speech vs Direct Speech Reported speech summarises
DIRECT INDIRECT SPEECH IMPERATIVES DIRECT SPEECH Direct speechDIRECT INDIRECT SPEECH IMPERATIVES DIRECT SPEECH Direct speech
Reported Speech vs Direct Speech Reported speech summarisesReported Speech vs Direct Speech Reported speech summarises
SPEECH ITS NATURE AND IMPORTANCE 1 SPEECH SpeechSPEECH ITS NATURE AND IMPORTANCE 1 SPEECH Speech
Speech Compression Speech Compression objectives Speech compression refersSpeech Compression Speech Compression objectives Speech compression refers
Speech Recognition Principles Speech Recognition Concepts Speech recognitionSpeech Recognition Principles Speech Recognition Concepts Speech recognition
Jamies speech Jamies speech Topic Jamies speech TopicJamies speech Jamies speech Topic Jamies speech Topic
REPORTED SPEECH Reported Speech Reported speech kalimat tidakREPORTED SPEECH Reported Speech Reported speech kalimat tidak
PUBLIC SPEECH WORKSHOP IV Speech Skills Speech SkillsPUBLIC SPEECH WORKSHOP IV Speech Skills Speech Skills
Reported Speech AIM Speech What a person saysReported Speech AIM Speech What a person says
HTML HTML Hyper Text Markup Language The LanguageHTML HTML Hyper Text Markup Language The Language
Spoken Language Understanding Speech Understanding parsing recognition SpeechSpoken Language Understanding Speech Understanding parsing recognition Speech