Prosody Ling 575 Spoken Dialog April 30 2015

Prosody Ling 575 Spoken Dialog April 30, 2015

Roadmap Defining prosody Why prosody? Prosody in dialog Quick introduction to To. BI

Defining Prosody Phonetic phenomena in speech than span more than a single segment-“suprasegmental” Prosody includes: Stress, focus, tone, intonation, length/pause, rhythm Prosodic features include: Pitch: perceptual correlate of fundamental frequency f 0: rate of vocal fold vibration Loudness/intensity, duration, segment quality

Why Prosody? Prosody plays a crucial role At all levels of language Lexical, syntactic, pragmatic/discourse Establishes meaning Disambiguates sense and structure Across languages families Common physiological, articulatory basis In synthesis and recognition of fluent speech

Prosody and the Lexicon Lexical: Determines word identity Prosodic effect at the syllable level (minimal unit) Lexical stress: syllable prominence Combination of length, pitch movement, loudness REcord (N) vs re. CORD (V)

Prosody and the Lexicon Lexical: Determines word identity Prosodic effect at the syllable level (minimal unit) Lexical tone: tone languages, e. g. Chinese, Punjabi Pitch height (register) and/or shape (contour) Ma (high): mother Ma (rising): hemp Ma (low): horse Ma (falling): scold

Intonation Ladd (1996) “Intonational Phonology” Definition: The use of suprasegmental features to convey sentence level pragmatics. ”

Three Aspects of Prosody Structure/boundaries: Sentences have prosodic structure Some words group together Others have noticeable break between them Prominence: Some words or syllables more prominent than others Tune: Intonational meolody over the utterance Jurafsky & Martin, 2008

Prosody and Syntax Prosody can disambiguate structure Associated with chunking and attachment Not identical with syntactic phrase boundaries “Prosody is predictable from syntax, except when it isn’t” Prosodic phrasing: Breaks utterances into phrases Can be guided by punctuation, but not fully Indicated by combination of pause, change in pitch

Chunking, or “phrasing” A 1: I met Mary and Elena’s mother at the mall yesterday. A 2: I met Mary and Elena’s mother at the mall yesterday. Example from Jennifer Venidetti

Punctuation & Prosody Humor A panda goes into a restaurant and has a meal. Just before he leaves he takes out a gun and fires it. The irate restaurant owner says ‘Why did you do that? ’ The panda replies, ‘ I'm a panda. Look it up. ’The restaurateur goes to his dictionary and under ‘panda’ finds: ‘black and white arboreal, bear like creatures; eats, shoots and leaves. ’

Prosody in Pragmatics & Discourse Prosodic prominence: Pitch accents Associated with focus, emphasis, information structure Acoustically: Associated with maxima/minima in f 0 contour Louder, Longer Often higher pitch “October eleventh”:

Which word receives an accent? It depends on the context. For example, the ‘new’ information in the answer to a question is often accented, while the ‘old’ information usually is not. Q 1: What types of foods are a good source of vitamins? A 1: LEGUMES are a good source of vitamins. Q 2: Are legumes a source of vitamins? A 2: Legumes are a GOOD source of vitamins. Q 3: I’ve heard that legumes are healthy, but what are they a good source of ? A 3: Legumes are a good source of VITAMINS. Speech and Language Processing Jurafsky and Martin Slide from Jennifer Venditti 9/3/2021 13

Prosody in Pragmatics & Discourse Sentence type, dialogue act: Statement vs. declarative question : “It’s raining (? )” from Shih, Prosody Learning and Generation Discourse Structure (Topic), Emotion

Yes-No question are legumes a good source of VITAMINS Rise from the main accent to the end of the sentence. Speech and Language Processing Jurafsky and Martin Slide from Jennifer Venditti 9/3/2021 15

‘Surprise-redundancy’ tune [How many times do I have to tell you. . . ] legumes are a good source of vitamins Low beginning followed by a gradual rise to a high at the end. Speech and Language Processing Jurafsky and Martin Slide from Jennifer Venditti 9/3/2021 16

‘Contradiction’ tune “I’ve heard that linguini is a good source of vitamins. ” linguini isn’t a good source of vitamins [. . . how could you think that? ] Sharp fall at the beginning, flat and low, then rising at the end. Speech and Language Processing Jurafsky and Martin Slide from Jennifer Venditti 9/3/2021 17

To. BI Commonly used intonation inventory for English Also adapted to other languages

To. BI Commonly used intonation inventory for English Also adapted to other languages

Challenges in Prosody I Highly variable Actual realization differs from ideal Speaker variation: Gender, vocal track differences, idiosyncrasy Tonal coarticulation Neighboring tones influence (like segmental) Underlying fall can become rise Parallel encoding Effects at multiple levels realized simultaneously

Challenges in Prosody II Challenges for learning Lack of training data Sparseness: Many prosodic phenomena are infrequent E. g. , non-declarative utterances, topic boundaries, contrastive accents, etc Challenging for machine learning methods Costs of labeling: Many prosodic events require expert labeling Need large corpus to attest Time-consuming, expensive