MULTIMODALITY AND GENRE CLUSTERING WORDS AND IMAGES IN

MULTIMODALITY AND GENRE : CLUSTERING WORDS AND IMAGES IN TEXTS Fauziah Taib Faculty of Languages and Linguistics University of Malaya Kuala Lumpur, Malaysia. 1

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Multimodality and Genre : Clustering Words and Images in Texts Five parts of the presentation : 1. 2. 3. 4. 5. Introduction : Multimodality and genre The cluster analysis framework An example of cluster analysis A system network for the genre of business-tobusiness brochures Summary 2

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Multimodal Texts that use multi-semiotic modes such as language, images, layout, colours, typography and these modes are integrated in the meaning making of the texts. l l All meaning making practices like early spelling, classroom interaction and scientific papers are multimodal for they deploy more than one mode of communication. (Kress, 1997; Kress and van Leeuwen, 2001; Lemke 1998). 3

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Images have meaning potential contact subjectivity social distance 4

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. The meaning of composition / layout Margin Ideal New Margin Ideal Given centre Margin Real Given Margin Real New (Kress & van Leeuwen, 1996) 5

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Weight, expansion, slope, curvature, connectivity, orientation etc 6

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Colours have meaning potential Warm colours Cold colours Brightness, saturation, purity, modulation, hue, luminosity 7

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Genre in Systemic Functional Linguistics (SFL) Genre, in SFL, is seen as a staged, goal-oriented social process (Martin, 1992) meaning each stage in the genre contributes with one part of the overall that must be made for the genre to accomplish its aims successfully. Research in this area sought to explicate the distinctive stages, moves of genres together with patterns of lexical, grammatical, and cohesive choices which construct the function of the stages of the genre (Rothery, 1996). 8

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Why the SFL approach to genre theory? l Despite focusing on language, the principles of SFL reflect the awareness of the fact that …there are other modes of meaning, in any culture, which are outside the realm of language. (Halliday and Hasan, 1985) l It has a systematic model of grammar which makes extensive use of a stratified, metafunctional grammar. (Hyland, 2002) 9

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Example of genre analysis Text : magazine advice column Functional label I lied on my cv Revealing a problem (confession) Should I come clean with my boss Appealing for help (question) Yes Providing a solution (answer) But prepared for the possibility of losing your job if you have a scrupulous Boss Issuing a caveat (warning) The bright side: you will gain her respect if you speak up and accept your mistake Predicting the result (prediction 1) and having got this burden off your chest will help you focus better on your work (prediction 2) Overall communicative purpose : Solution to a problem Multimodal realisations L a n g u a g e T y p o g r a p h y (van Leeuwen, 2005) 10

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Cluster Analysis in Multimodal Texts (Baldry and Thibault, 2006) Analyzing clusters which refer to a local grouping of items which may be verbal and/or visual in nature and which are spatially proximate thereby defining a specific region or subregion of the page as a whole. Clusters are functionally related to each other. 11

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. The Law of Figure-Ground Discrimination The Law of Closure The Law of Proximity The Law of Similarity Psychology Gestalt & many more… 12

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. 13

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Business-to-business texts l texts produced to advertise products or services to other businesses. l texts are all about disseminating information for “content is king for business-to-business marketing” (Murphy, 2007) unlike business-to-consumer brochures which appeal to the emotions of the individual customers (Kotler et al. 2002). 14

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. 6 - step procedure in analysing large sample of multimodal texts. 1. Clustering 2. Refining the clusters 3. Identifying functions of clusters 4. Labelling clusters 5. Grouping the clusters 6. Identifying major clusters 15

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Step 2 : Refining the title cluster new cluster 16

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Other parameters of clustering ? Local groupings based on spatial proximity. Local groupings based on lexical connectivity. Local groupings based on contents of items. 17

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Step 3 : Labelling cluster Step 4 : Identifying function(s) No. Clusters Function(s) Multimodal realisations 1. Detail state date, venue and/or time and/or name of facilitator -Language -typography -layout etc 2. Course highlights state contents of programme -Language -typography -layout etc 3. Training provider information i. identify the company organizing the programme ii. identify corporate identity -Language -typography -layout etc 4. Call and contact provide contact details -Language -Typography -layout etc 18

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Step 5 : Grouping clusters Clusters (26) Title Detail Methodology resource material Content Focus Target audience training provider information Registration Call and contact testimony Fee Introduction Purpose Objective Benefit Illustration Focuser catch phrase complimentary item certificate of completion discounts and savings claimable fee facilitator’s profile training company’s profile accreditation Step 6 : Identifying major cluster Groups (9) Major clusters Detail Methodology Resource material Content Focus Target audience Programme information major cluster Registration Call and contact Response solicitation major cluster Testimony Fee Value major cluster Introduction Purpose Objective Benefit Justifier major cluster 19

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Major clusters of business-to-business texts 1. 2. 3. 4. 5. 6. 7. 8. 9. Title Programme information Signature Value Justifier Attention- getter Incentive Credibility Response solicitation 20

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. System network for the generic structure of Malaysian business-to-business brochures This would be one of the earliest attempts to establish a system network for a genre. The syntagmatic and the paradigmatic relations of elements can be presented in what is called as the system (Eggins, 2004). 21

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. SYSTEM NETWORK FOR THE GENERIC STRUCTURE OF BUSINESS-TO-BUSINESS TEXTS 22

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Syntagmatic relations of major clusters Title Value Solicit response Programme information Paradigmatic choices Realisations of each cluster type One-cluster Title One-cluster Testimony Fee Two-cluster Registration Call and contact Registration + Call and contact Two-cluster Detail + Content One-cluster Three-cluster Four-cluster Detail + Content + Target audience Detail + Content + Methodology Detail + Content + Target audience + Course highlight 23

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. Summary 1. Cluster analysis + SF Multimodal discourse analysis 2. Additional parameters to be considered in cluster analysis framework 3. A 6 – step procedure in cluster analysis to analyse large corpus of multimodal texts 4. A system network for the generic structure of business advertising texts. 24

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. References 1. Baldry, A. and P. Thibault. 2006. Multimodal Transcription and Text Analysis. London/Connecticut: Equinox Publishing Ltd. 2. Eggins, S. 2004. An Introduction to Systemic Functional Linguistics. (2 nd Edition). London/New York: Continuum. 3. Halliday, M. A. K. and R. Hasan. 1985. Language, Context and Text : Aspects of Language in a Social-Semiotic Perspective. Geelong, Australia: Deakin University Press. 4. Kress, G. and T. van Leeuwen. 1996. Reading Images : The Grammar of Visual Design. London/New York: Routledge. 5. Van Leeuwen, T. 2005. Introducing Social Semiotics. London/New York: Routledge. 25

Multimodality and Genre : Clustering Words and Images in Texts. 12 th ESEA Conference. 12 th-14 th Dec. 2007. KMUTT, Bangkok, Thailand. THANK YOU Fauziah Taib Faculty of Languages and Linguistics University Malaya 50603 Kuala Lumpur Malaysia. fauziaht@um. edu. my 26