Lecture 08 Semiotic Media Theory IS 246 Multimedia

Lecture 08: Semiotic Media Theory IS 246 Multimedia Information Prof. Marc Davis UC Berkeley SIMS Monday and Wednesday 3: 30 pm – 5: 00 pm Fall 2003 http: //www. sims. berkeley. edu/academics/courses/is 246/f 04/ IS 246 - FALL 2004. 09. 27 - SLIDE 1

Today’s Agenda • Review of Last Time – Editing II • Semiotic Media Theory – The Video Sign – Cinematic Articulations – Syntagmatic Structures • Discussion Questions • Action Items for Next Time IS 246 - FALL 2004. 09. 27 - SLIDE 2

Today’s Agenda • Review of Last Time – Editing II • Semiotic Media Theory – The Video Sign – Cinematic Articulations – Syntagmatic Structures • Discussion Questions • Action Items for Next Time IS 246 - FALL 2004. 09. 27 - SLIDE 3

Kuleshov Effect • Kuleshov Effect – Neutral Face Soup (“Pensive” Face) – Neutral Face Dead Woman (“Sad” Face) – Neutral Face Child playing with toy bear (“Happy” Face) – How do you describe the face? • Video has a dual semantics – Sequence-independent stable semantics of shots – Sequence-dependent variable semantics of shots IS 246 - FALL 2004. 09. 27 - SLIDE 4

Isenhour on Context and Order • A 1: B; A 2: C A 1 NOT EQUAL A 2 – Shot context affects shot meaning • The shot before affects the shot after • B: A 1; C: A 2 A 1 NOT EQUAL A 2 – Shot context affects shot meaning • The shot after affects the shot before • A: B NOT EQUAL B: A – Short order effects shot meaning • (A: B): C NOT EQUAL A: (B: C) IS 246 - FALL 2004. 09. 27 - SLIDE 5

Burch’s Transitions • Temporal transitions – Continuous – Discontinuous • Temporal ellipsis – Measurable time ellipsis – Indefinite time ellipsis • Temporal reversal (flashback, overlapping cut) – Measurable time reversal – Indefinite time reversal IS 246 - FALL 2004. 09. 27 - SLIDE 6

Burch’s Transitions • Spatial transitions – Continuous – Discontinuous • Proximal • Radically discontinuous IS 246 - FALL 2004. 09. 27 - SLIDE 7

Barthes’ Action Sequences • Consecutive – Temporal succession • Consequential – Causal succession • Volitive – Action results from an act of will • Reactive – Causal succession based on stimulus-response • Durative – Indicating the beginning, ending, or duration of an action • Equipollent – Necessarily paired actions (e. g. , asking a question and answering a question) IS 246 - FALL 2004. 09. 27 - SLIDE 8

Today’s Agenda • Review of Last Time – Editing II • Semiotic Media Theory – The Video Sign – Cinematic Articulations – Syntagmatic Structures • Discussion Questions • Action Items for Next Time IS 246 - FALL 2004. 09. 27 - SLIDE 9

Semiotic Media Theory • Application of semiotic methods of analysis to media, especially cinema • Traditionally used for structural and functional analysis • In computational media (especially automatic film editing), used also to define the primitives, means of combination, and means of abstraction for cinematic synthesis IS 246 - FALL 2004. 09. 27 - SLIDE 10

Semiotic Media Theory • Key questions for semiotics of cinema – How does cinema mean? – What are the significant structures of cinematic form? – What are the various hierarchical levels of these structures? – How do they compare to linguistic structures? – Does cinema have any unique semiotic properties? IS 246 - FALL 2004. 09. 27 - SLIDE 11

Today’s Agenda • Review of Last Time – Editing II • Semiotic Media Theory – The Video Sign – Cinematic Articulations – Syntagmatic Structures • Discussion Questions • Action Items for Next Time IS 246 - FALL 2004. 09. 27 - SLIDE 12

The Signified Signifier IS 246 - FALL 2004. 09. 27 - SLIDE 13

The Linguistic Sign “dog” dog IS 246 - FALL 2004. 09. 27 - SLIDE 14

The Video Sign “dog” IS 246 - FALL 2004. 09. 27 - SLIDE 15

Arbitrariness of the Video Sign • Theories of video denotation – Iconic (i. e. , onomatopoetic) • Video is a mechanical replication of what it represents – Arbitrary • Video constructs an arbitrary relationship between signifier and signified – Motivated • The relationship between the signifier and signified is motivated, but by what? – A “natural” analogy between video and the world? – By the conventions of cinematic language? IS 246 - FALL 2004. 09. 27 - SLIDE 16

Today’s Agenda • Review of Last Time – Editing II • Semiotic Media Theory – The Video Sign – Cinematic Articulations – Syntagmatic Structures • Discussion Questions • Action Items for Next Time IS 246 - FALL 2004. 09. 27 - SLIDE 17

Articulation • Articulation – Any form of semiotic organization which produces distinct combinable units • Double articulation in natural language – First articulation • Morphemes: smallest formal units of significance (e. g. , “cow”) • Constructed out of phonemes – Second articulation • Phonemes: sound units which in and of themselves lack significance (e. g. , “c” “ow”) IS 246 - FALL 2004. 09. 27 - SLIDE 18

Commutation • Etymologically “change together” • The substitution of one signifier for another produces a change of the signified • Example in phonemes to morphemes – Different pronunciations of the “ow” in “cow” will still be understood as “cow” – But we distinguish “cow” “caw” “quay” “coo” “cal” IS 246 - FALL 2004. 09. 27 - SLIDE 19

Cinematic Articulations • Metz – Cinema has no double articulation because its smallest units (“shots”) are significant – Based on Bazinian view of cinema as reproduction of reality • Eco – Cinema has three levels of articulation which include sub-shot units – Similar to Eisensteinian view of cinema as construction of representations IS 246 - FALL 2004. 09. 27 - SLIDE 20

Iconic Figures / Semes / Signs Iconic Figures (deduced from perceptive codes) constituting a paradigm from which units are selected to form… Iconic Signs IS 246 - FALL 2004 Iconic Semes Photograms 2004. 09. 27 - SLIDE 21

Eco’s Photographic Articulations • Iconic semes – Example: “a dark-haired man stands here wearing a patterned shirt” • Iconic signs – Example: human nose, human eye, shirt, etc. • Iconic figures – Example: angles, light contrasts, curves, etc. IS 246 - FALL 2004. 09. 27 - SLIDE 22

Eco’s Cinematic Articulations • Kinesic semes (kinemorphs) – Example: “I’m saying yes to the person on the right” • Kinesic signs (kines) – Example: Nod head yes • Kinesic figures – Example: move head to right, move head up, move head down IS 246 - FALL 2004. 09. 27 - SLIDE 23

Kinesic Figures / Kinemorphs Iconic Figures Kinemorphs Kinesic Figures Iconic Signs Iconic Semes Diachrony Photograms Synchrony IS 246 - FALL 2004. 09. 27 - SLIDE 24

Today’s Agenda • Review of Last Time – Editing II • Semiotic Media Theory – The Video Sign – Cinematic Articulations – Syntagmatic Structures • Discussion Questions • Action Items for Next Time IS 246 - FALL 2004. 09. 27 - SLIDE 25

Metz’s Grand Syntagmatique Autonomous Shot Autonomous Segments Achronological Syntagmas Parallel Syntagma Bracket Syntagmas Chronological Syntagmas Descriptive Syntagma Alternate (Narrative) Syntagma Narrative Syntagmas Scene Linear (Narrative) Syntagmas Episodic Sequences Ordinary Sequence IS 246 - FALL 2004. 09. 27 - SLIDE 26

Metz’s Grand Syntagmatique • Autonomous Shot (single shot) – Single-Shot Sequence (complete unto itself) – Inserts (differentiated from shot context) • Non-diegetic insert – A single shot which presents objects exterior to the story world • Displaced diegetic insert – Diegetic images temporally and/or spatially out of context • Subjective insert – Memories, fears, dreams, etc. of character • Explanatory insert – Single shots which clarify diegetic events IS 246 - FALL 2004. 09. 27 - SLIDE 27

Metz’s Grand Syntagmatique • Achronological Syntagmas – Parallel Syntagma (alternating) • Two alternating motifs without clear spatial or temporal relationship – Bracket Syntagma (non-alternating) • Brief scenes without temporal sequence but often organized around a concept • Chronological Syntagmas – Descriptive Syntagma (non-narrative) • Objects shown to create spatial contiguity to situate action – Narrative Syntagmas (narrative) IS 246 - FALL 2004. 09. 27 - SLIDE 28

Metz’s Grand Syntagmatique • Narrative Syntagmas – Alternate (Narrative) Syntagma • Narrative crosscutting showing temporal simultaneity (“parallel action”) – Linear (Narrative) Syntagma • Scene (continuous) – Spatial contiguity and temporal continuity across a series of shots • Sequences (elliptical) – Episodic Sequence » Symbolic summary of chronological progression usually to compress time (“montage sequence”) – Ordinary Sequence IS 246 - FALL 2004. 09. 27 - SLIDE 29

Brian O’Connor on Metz IS 246 - FALL 2004. 09. 27 - SLIDE 30

Today’s Agenda • Review of Last Time – Editing II • Semiotic Media Theory – The Video Sign – Cinematic Articulations – Syntagmatic Structures • Discussion Questions • Action Items for Next Time IS 246 - FALL 2004. 09. 27 - SLIDE 31

Discussion Questions (Barthes) • Jaiwant Virk on Barthes – If “The principle of narrative art… it is a matter of producing a discourse which best satisfies the demand for completeness…”, then how does one define the boundaries of the action sequence? How much cultural history does one pack to prevent the “horror of the vacuum”? Or should the sequence be completely abandoned to a subjective analysis of the reader/viewer? IS 246 - FALL 2004. 09. 27 - SLIDE 32

Discussion Questions (Barthes) • Jaiwant Virk on Barthes – What are the possible factors that can be considered to define/label an action sequence for media metadata purpose (motion, etc. )? IS 246 - FALL 2004. 09. 27 - SLIDE 33

Discussion Questions (Eco) • Yongwook Jeong on Eco – According to Eco, because all signs are arbitrary and conventional, we have to learn how to interpret them. This made me have a question about whether it is possible to have more than one languages of cinema. Does this make it possible to distinguish one kind of films from another? IS 246 - FALL 2004. 09. 27 - SLIDE 34

Discussion Questions (Eco) • Yongwook Jeong on Eco – Regarding the cinematic code as the only code carrying a triple articulation, Eco seems to make iconic codes a basis for his cinematic language. Can we apply his arguments to today's films? IS 246 - FALL 2004. 09. 27 - SLIDE 35

Discussion Questions (Metz) • Ryan Shaw on Metz – Metz believes that in cinema, unlike in spoken language, even the smallest identifiable units are complete signs, having a signifier (the image) and a signified (the thing which the image is depicting). Moreover, unlike in linguistic signs, the relationship between the signifier and the signified in these signs is motivated by perceptual similarities ("the image of a dog is like the dog"). – What implications does Metz's view have for the task of programming a computer to understand a film? If the smallest analyzable units of cinema depend on human perception for their meaning, must we solve the problems of computational perception before such a task can be undertaken? Or might the perceptual motivations of cinematic signs be encoded in metadata in such a way that a computer could construct an understanding of a film, even without the ability to perceive the motivations itself? IS 246 - FALL 2004. 09. 27 - SLIDE 36

Discussion Questions (Metz) • Ryan Shaw on Metz – Metz struggles with the application of linguistic models to the question of how films are understood. His comparisons of linguistics and cinema seem to point out more differences than similarities, yet he persists in his effort due to his belief that “the methods of linguistics. . . provide the semiotics of cinema with a constant and precious aid in establishing units that. . . are liable over time. . . to become progressively refined. ” – Is this belief justified? Is the semiotics of cinema making progress in refining these units? Or might it be more useful to stop trying to treat cinema as a “language” and investigate other methods (psychological or neurological, perhaps)? IS 246 - FALL 2004. 09. 27 - SLIDE 37

Today’s Agenda • Review of Last Time – Editing II • Semiotic Media Theory – The Video Sign – Cinematic Articulations – Syntagmatic Structures • Discussion Questions • Action Items for Next Time IS 246 - FALL 2004. 09. 27 - SLIDE 38

Metz on Shot/Word Distinction • • • Shots are infinite in number, contrary to words, but like statements, which can be formulated in verbal language. Shots are the creations of the film-maker, unlike words (which pre-exist in lexicons), but similar to statements (which are in principle the invention of the speaker). The shot presents the receiver with a quantity of undefined information, contrary to the word. From this point of view, the shot is not even equivalent to the sentence. Rather, it is like the complex statement of undefined length (how is one to describe a film shot completely by means of natural language? ). The shot is an actualized unit, a unit of discourse, an assertion, unlike the word (which is a purely virtual lexical unit), but like the statement, which always refers to reality or a reality (even when it is interrogative or jussive). The image of a house does not signify “house, ” but “Here is a house”; the image contains a sort of index of actualization, by the mere fact that it occurs in a film. Only to a small extent does a shot assume its meaning in paradigmatic contrast to the other shots that might have occurred at the same point along the filmic chain (since the other possible shots are infinite in number), whereas the word is always part of at least one more or less organized semantic field. (Metz 1974: 115 -116). IS 246 - FALL 2004. 09. 27 - SLIDE 39

Administrivia • You MUST have a SIMS computer account to do Assignment 1 IS 246 - FALL 2004. 09. 27 - SLIDE 40

Readings for Next Time • Wednesday 09/29 – Eisenstein, S. M. Film Form: Essays in Film Theory. Harcourt Brace Jovanovich, Publishers, San Diego, 1949; pp. 45 -63. IS 246 - FALL 2004. 09. 27 - SLIDE 41