The Constituents of Modern Digital Communication 1 The

The Constituents of Modern Digital Communication 1

The Constituents • Hypertextuality • (Affective) Interactivity • Multimodality 2

Hypertextuality • The Greek prefix 'hyper' stands for 'above', 'beyond', 'outside'; hence, by hypertext we mean a text that refers to something above, beyond, and outside itself in order to reach its status • This neologism, coined by Ted Nelson in the 1960 s, perfectly describes the real transformation from "tactile" to "digital" expression substance, according to Baudrillard's terminology (1983). 3

Hypertextuality • “Text composed of blocks of words (or images) linked electronically by multiple paths, chains or trails in an open-ended, perpetually unfinished textuality described by the terms link, node, network, Web, and path” (Landow, 1992: 3) • “By 'hypertext' I mean nonsequential writing text that branches and allows choices to the reader, best read at an interactive screen. "(Nelson, 1981: 0/2). 4

Hypertextuality Qualities of hypertexts (Bettetini et al. , 1999): • Multilinearity • Nonsequentiality • Granularity • Connectivity • Reticularity • Interactivity > All these features are qualities of hypertexts, but what allows them to be actualised is the presence of LINKS 5

Hypertextuality • Multilinearity and Nonsequentiality: we have blocks of text that can be combined together following different paths. • These two features do not exclude the sequential and linear structure of written words. This can remains stable inside each minimal unit of information (node, chunk, lexia). 6

7

Figure 1 • This screen-shot shows, as its largest element, a block of writing of a ‘traditional’ kind: an arrangement which is characteristic, in part at least, of a ‘traditional social order’ untouched by the technological potentials of the computer or mobile technologies. 8

9

Figure 2 • By contrast, this second figure shows an arrangement of entities of various kinds that has little or no resemblance to the page of a traditional book. It shows anything but a traditional written element; writing is not dominant, nor does linearity dominate. Here, linearity is replaced by modularity. 10

Linearity • Socially speaking, the formal / semiotic feature of linearity correlates with and ‘materializes’ the social feature of authority. • It points to how the text was made and by whom; and its arrangement tells the reader how to read the text (where to start reading, how to continue reading) and, through that, tells how to ‘get the’ meanings of the 11 author.

Linearity • Authority and authorship are entirely intertwined. ‘Accurate’ access to the meanings of the author depends on the reader’s following the implicit instructions on how to read this text: an instruction to follow a specific ‘reading path’. 12

Linearity • Linearity, here, is the sign of social power of a certain kind. • This has led to a ‘naturalized’ order of engaging with a text, described by the metaphor of a ‘reading path’. • Such reading paths are signs of the authority of the author. 13

Modularity • Modularity (i. e. ‘modules’ as the compositional elements) is a formalsemiotic feature that derives from social arrangements of a certain kind, and expresses and reflects social meanings. • They point to processes of text-making, not usually by a single author, but by a design-team and its practices. 14

Modularity • Modularity also points to the assumed manner of engaging with or ‘reading’ the text: not by following an order established by an author, but according to the interest of the person who engages with the text. 15

Modularity • In making the text, a design-team reflect on the characteristics of those who might ‘visit’ the ‘site’; make assumptions about their aesthetic dispositions (reflected in the colour-palette, for instance - as in the drawing and painting style); • They assume that users expect to exercise ‘choice’: that is, to be able to move ‘across’ this ‘collage-like’ text and make selections 16 according to their interest.

Modularity • Modularity then inverts the social and power relations of maker and reader. • While linearity insists that the ordering of the author has to be observed, modularity makes no such assumptions and demands. 17

Modularity • Modularity insists that the interest of the visitor of this site is preminent. • It rests on a different distribution of responsibilities, namely that: – the task of the designer(s) is to assemble materials, contents, which will prove to be of interest to a reader, – the reader will then make their choice about where to enter the page, and, by doing that, make a decision about how to move through the website. 18

Hypertextuality • Granularity refers to the extension of the nodes. The material dimension of a lexia should be exactly the same as that of the screen as such: it is necessary that it correspond to the information unit in terms of content dimension. • In the case of large portions of text, each node has to be re-structured either by dividing it into multiple sub-nodes or by transforming it into a multimodal node by means of different nonverbal codes, such as audio, video, and icons. • The guarantee of internal coherence and semantic autonomy in each lexia is the fundamental requisite. 19

Hypertextuality • Connectivity, or "secondary sequentiality“: how diverse portions of content are connected in a hypertext during individual navigations. • This aspect is twofold as it involves both authors and users. Consuming a hypertext not only means reading it, but also navigating it: two cognitive skills (Garzone, 2007). • Textual cohesion and coherence in each node depend on the cognitive ability and meaning-making competence of the author in the whole hypertext project: both users (primarily) and authors (secondarily) are responsible for these two principles. 20

Hypertextuality • Reticularity is a sort of hyper-connectivity because it is embedded in the outer environment of each hypertext, i. e. it is formed by those external links which lead to other stand-alone hypertexts or to other pages on the Web. • Reticularity belongs to hypertextuality just as intertextuality belongs to textuality. 21

Interactivity • Interactivity is actually the pre-requisite for connectivity and reticularity since it allows them to be realised. Without any possibility of interaction between. • Hypertext spaces are interactive by nature, they are conceived to have multiple readers/users who participate in the actualisation process: Travelling a hypertext is an act of both hyperdecoding and hypercoding. • These processes are unlimited if they occur in non stand-alone systems and if external links are present in the hypertext architecture. 22

Interactivity • This happens by means of Graphical User Interface (GUI) design, graphics, multimedia and so on, but at the base of these technologies there is the coherent use of internal and/or external links. • Hypertexts and Web pages are not simple technologies but rather technologies of signification. • A creator of links conceives of them as meaning potential, whereas users of links perceive them as meaning actuality before and after navigating them (see ch. 5). 23

Multimodality • The correct definition of a digital product should be "hypermedia" (Landow 1992, Bolter 2001), rather than hypertext, since hypertext seldom exists without any graphics or without any other media intervention. • Due to the widespread use of print, verbocentricity has played an exclusive role, by empowering the verbal code to play the role of meaning container. 24

Multimodality • What has changed after the advent of digital writing is the relationship between graphics and verbal text. • The Greek rhetoric technique, called ekphrasis, aimed to rival pictorial, sound or sensory experiences in words, recognising in them the task/capacity of perfectly describing and explaining what is seen, heard and felt. 25

Multimodality • In modern communication, the presence of images (e. g newspapers) as well as sounds and animations (e. g. films or videos) reverses this equilibrium: "we get a reverse ekphrasis in which images are given the task of explaining words" (Bolter, 2001: 56). • By contrast, in multimodal communication no semiotic resource prevails or can live separately from the others: all the resources that are conjointly employed co-operate within a flat hierarchy and are equally responsible for meaning creation. 26

Multimodality • A clear distinction between multimediality and multimodality is necessary. • They are not synonymous because their objects of study are different: the former deals with media, the latter with modes. • Media are the material resources used in production, including tools and materials. • Modes are socially and culturally shaped semiotic resources (codes) for making meaning, and they intervene in the realisation of types of (inter)action. 27

Multimodality • Meaning construction can take place simultaneously by working through different domains of practice, named strata (Kress&Van Leeuwen, 2001): – – Discourse/content form Design/content substance Production/expression form Distribution/expression substance. • A message is no longer developed according to the traditional double articulation (form and content) but according to a multiple articulation which coordinates all the modes involved and guarantees coherence among strata 28

Multimodality • The concept of discourse refers to: “[…] socially situated forms of knowledge about (aspects of) reality. This includes knowledge of events constituting that reality (who is involved, what takes place, where and when it takes place, and so on) as well as a set of related evaluations, purposes, interpretations and legitimations. ” 29

Multimodality • the concept of design refers to “[…] means to realise discourse in the context of a given communication situation. […] But design is still separate from the actual material of production of the semiotic product or the actual material articulation of the semiotic event. The resources on which design draws, the semiotic modes, are still abstract, capable of being realised in different materialities. ” 30

Multimodality • Production is “the communicative use of media, of material resources. […] Production is always physical work, whether by humans or machines, a physical job of articulating 'text'. And the interpretation of production is also a physical work, a use of the body (sensory organs). […] meaning does not only reside in discourse and design, it also resides in production. It results from human engagement with the world, and the resources we use in articulating and interpreting meaning comprise both semiotic modes and semiotic media. ” 31

Multimodality • Distribution "refers to the technical 're-coding' of semiotic products and events, for purposes of recording (e. g. tape recording, digital recording) and/or distribution (e. g. telephony, radio, and television transmission). " • These two peculiarities, re-coding as recording and re-coding as transmitting, determine the meaning potential of this stratum. 32

Multimodality • This is due to the fact that the medium by which a text is recorded is different from the original that produces it, just as time, space and the participants of the event can be different whenever the text is consumed. • Every act of representation, realised by means of different material resources, entails transformation to the extent to which it migrates from one medium into another, from one consuming process into others: this is the real nature of semiosis. 33