ICS 218 Multimedia Systems and Applications Lecture 1

ICS 218 -Multimedia Systems and Applications Lecture 1 - Introduction to Multimedia Prof. Nalini Venkatasubramanian nalini@ics. uci. edu Introduction to Multimedia 1

Course logistics and details z Course Web page yhttp: //www. ics. uci. edu/~ics 218 z Lectures - MW 2: 00 -3: 20 p. m, ELH 110 z Course Laboratories - machines on 3 rd floor CS labs z ICS 218 Textbook: Multimedia: Computing, Communications and Applications Ralf Steinmetz and Klara Nahrstedt, Prentice-Hall Inc. z Other reading material x. Technical papers and reports Introduction to Multimedia 2

Course logistics and details z Homeworks and Assignments y 3 homeworks in the quarter of which 1 or 2 may be programming assignments. z Tests y. Final Exam - as per UCI course catalog z Course Project y. Maybe done individually, in groups of 2 or 3(max) y. Potential projects on webpage x. Data representation, multimedia systems and networks, multimedia applications Introduction to Multimedia 3

ICS 218 Grading Policy z Homeworks - 30% • (3 homeworks each worth 10% of the final grade). z Class Project - 40% of the final grade z Final exam - 30% of the final grade z Final assignment of grades will be based on a curve. Introduction to Multimedia 4

Lecture Schedule y. Weeks 1 and 2: x. Multimedia Representation • Introduction to Multimedia • Audio/Image/Video Representation y. Weeks 3 and 4: x. Multimedia Compression • Encoding and Compression Techniques • Image Compression (JPEG) • Video Compression (MPEG/MPEG 2/MPEG 4) y Week 5 and 6: x. Multimedia Resource Management • Multimedia Quality of Service and Server Design • Multimedia Operating Systems - Process Management, Filesystems and buffer management Introduction to Multimedia 5

Course Schedule y. Weeks 7 and 8: x. Multimedia Communication Systems • MM Networking • MM Communication y. Week 9: x. Multimedia Synchronization • Multimedia Synchronization Concepts • Multimedia Synchronization Enforcement y. Week 10: x. Multimedia Applications and Services • Multimedia Video Conferencing • Multimedia Entertainment Applications Introduction to Multimedia 6

Introduction z Multimedia Description z Why multimedia systems? z Classification of Media z Multimedia Systems z Data Stream Characteristics Introduction to Multimedia 7

Multimedia Description z Multimedia xis an integration of continuous media (e. g. audio, video) and discrete media (e. g. text, graphics, images) through which digital information can be conveyed to the user in an appropriate way. y. Multi xmany, much, multiple y. Medium x. An interleaving substance through which something is transmitted or carried on Introduction to Multimedia 8

Why Multimedia Computing? y. Application driven xe. g. medicine, sports, entertainment, education y. Information can often be better represented using audio/video/animation rather than using text, images and graphics alone. y. Information is distributed using computer and telecommunication networks. y. Integration of multiple media places demands on xcomputation power xstorage requirements xnetworking requirements Introduction to Multimedia 9

Multimedia Information Systems z Technical challenges y. Sheer volume of data x. Need to manage huge volumes of data y. Timing requirements xamong components of data computation and communication. x. Must work internally with given timing constraints - real-time performance is required. y. Integration requirements xneed to process traditional media (text, images) as well as continuous media (audio/video). x. Media are not always independent of each other synchronization among the media may be required. Introduction to Multimedia 10

High Data Volume of Multimedia Information Introduction to Multimedia 11

Technology Incentive z Growth in computational capacity x. MM workstations with audio/video processing capability x. Dramatic increase in CPU processing power x. Dedicated compression engines for audio, video etc. z Rise in storage capacity x. Large capacity disks (several gigabytes) x. Increase in storage bandwidth, e. g. disk array technology z Surge in available network bandwidth xhigh speed fiber optic networks - gigabit networks xfast packet switching technology Introduction to Multimedia 12

Application Areas z Residential Services xvideo-on-demand xvideo phone/conferencing systems xmultimedia home shopping (MM catalogs, product demos and presentation) xself-paced education z Business Services x. Corporate training x. Desktop MM conferencing, MM e-mail Introduction to Multimedia 13

Application Areas z Education x. Distance education - MM repository of class videos x. Access to digital MM libraries over high speed networks z Science and Technology xcomputational visualization and prototyping xastronomy, environmental science z Medicine x. Diagnosis and treatment - e. g. MM databases that provide support for queries on scanned images, X-rays, assessments, response etc. Introduction to Multimedia 14

Classification of Media y. Perception Medium x. How do humans perceive information in a computer? • Through seeing - text, images, video • Through hearing - music, noise, speech y. Representation Medium x. How is the computer information encoded? • Using formats for representing and information • ASCII(text), JPEG(image), MPEG(video) y. Presentation Medium x. Through which medium is information delivered by the computer or introduced into the computer? • Via I/O tools and devices • paper, screen, speakers (output media) • keyboard, mouse, camera, microphone (input media) Introduction to Multimedia 15

Classification of Media (cont. ) y. Storage Medium • Where will the information be stored? • Storage media - floppy disk, hard disk, tape, CD-ROM etc. y. Transmission Medium • Over what medium will the information be transmitted? • Using information carriers that enable continuous data transmission - networks • wire, coaxial cable, fiber optics y. Information Exchange Medium • Which information carrier will be used for information exchange between different places? • Direct transmission using computer networks • Combined use of storage and transmission media (e. g. electronic mail). Introduction to Multimedia 16

Media Concepts z Each medium defines x. Representation values - determine the information representation of different media • Continuous representation values (e. g. electro-magnetic waves) • Discrete representation values(e. g. text characters in digital form) x. Representation space determines the surrounding where the media are presented. • Visual representation space (e. g. paper, screen) • Acoustic representation space (e. g. stereo) Introduction to Multimedia 17

Media Concepts (cont. ) z Representation dimensions of a representation space are: y. Spatial dimensions: xtwo dimensional (2 D graphics) xthree dimensional (holography) y. Temporal dimensions: x. Time independent (document) - Discrete media • Information consists of a sequence of individual elements without a time component. x. Time dependent (movie) - Continuous media • Information is expressed not only by its individual value but also by its time of occurrence. Introduction to Multimedia 18

Multimedia Systems z Qualitative and quantitative evaluation of multimedia systems y. Combination of media xcontinuous and discrete. y. Levels of media-independence xsome media types (audio/video) may be tightly coupled, others may not. y. Computer supported integration xtiming, spatial and semantic synchronization y. Communication capability Introduction to Multimedia 19

Data Streams z Distributed multimedia communication systems xdata of discrete and continuous media are broken into individual units (packets) and transmitted. z Data Stream xsequence of individual packets that are transmitted in a timedependant fashion. x. Transmission of information carrying different media leads to data streams with varying features • Asynchronous • Synchronous • Isochronous Introduction to Multimedia 20

Data Stream Characteristics x. Asynchronous transmission mode • provides for communication with no time restriction • Packets reach receiver as quickly as possible, e. g. protocols for email transmission x. Synchronous transmission mode • defines a maximum end-to-end delay for each packet of a data stream. • May require intermediate storage • E. g. audio connection established over a network. x. Isochronous transmission mode • defines a maximum and a minimum end-to-end delay for each packet of a data stream. Delay jitter of individual packets is bounded. • E. g. transmission of video over a network. • Intermediate storage requirements reduced. Introduction to Multimedia 21

Data Stream Characteristics y. Data Stream characteristics for continuous media can be based on x. Time intervals between complete transmission of consecutive packets • Strongly periodic data streams - constant time interval • Weakly periodic data streams - periodic function with finite period. • Aperiodic data streams x. Data size - amount of consecutive packets • Strongly regular data streams - constant amount of data • Weakly regular data streams - varies periodically with time • Irregular data streams x. Continuity • Continuous data streams • Discrete data streams Introduction to Multimedia 22

Classification based on time intervals Strongly periodic data stream T Weakly periodic data stream T 1 T 2 T 3 T Aperiodic data stream T 1 T T 2 Introduction to Multimedia 23

Classification based on packet size Strongly regular data stream D 1 t T D 1 Weakly regular data stream Irregular data stream t t D 1 D 2 D 3 T D 1 D 2 D 3 Dn Introduction to Multimedia 24

Classification based on continuity Continuous data stream D 1 D 2 D 3 D 4 D Discrete data stream Introduction to Multimedia 25

Logical Data Units y. Continuous media consist of a time-dependent sequence of individual information units called Logical Data Units (LDU). • a symphony consists of independent sentences • a sentence consists of notes • notes are sequences of samples y. Granularity of LDUs • symphony, sentence, individual notes, grouped samples, individual samples • film, clip, frame, raster, pixel y. Duration of LDU: • open LDU - duration not known in advance • closed LDU - predefined duration Introduction to Multimedia 26

Granularity of Logical Data Units Film Clip Frame Blocks Pixels Introduction to Multimedia 27