e Commerce Technology 20 751 Lecture 6 Multimedia

e. Commerce Technology 20 -751 Lecture 6: Multimedia

MULTIMEDIA • Which media can be delivered remotely? – Text, graphics, audio, video, motion picture, virtual reality • What are the limitations? – Speed, resolution, fidelity, color • How are they created? • How are they stored? – File formats: GIF, TIFF, JPEG, MPEG, . . . • How are they displayed (put in web pages)?

Pixels (Picture Cells) Discrete array of cells holding gray-scale data 1 -bit (black or white) 8 -bit (0 - 255) To represent color, use multiple pixel arrays (usually 3)

RGB (Additive) Color Space CREATING COLORS BY ADDING RED, GREEN + BLUE • • RGB SCREEN PHOSOPHORS R, G, B each indicated by intensity 0 -255 (24 bits) Additive color applet Visibone color selector RGB, RBW, HSV, RGYB, HLS, YIQ converter

Gamut (full range of colors) Gray = human visible range 1. WIDE-GAMUT RGB 2. HDTV (SM PTE 240 -M) 3. Custom RGB 4. Color. Match RGB 5. s. RGB (average monitor) CYM = cyan, magenta, yellow (subtractive)

YUV Color Space • Y = luminance (brightness, intensity of yellow) • U, V = chrominance (difference between a color and yellow) U = B - Y, V = R - Y • YUV is used in TV SOURCE: N. BARNETT, VA. TECH.

YUV Color Space Y = 0. 299 R + 0. 587 G + 0. 114 B U = 0. 492 (B - Y) V = 0. 877 (R - Y)

Bit Depth 16, 777, 216 COLORS 256 COLORS 16 COLORS 2 COLORS: B, W

Dithering • • File contains color information system can’t display Dithering is approximating a non-displayable color Image dithering: replace with 2 or more colors Screen dithering (substitution): replace with 1 color IMAGE-DITHERED VERSION OF MARKED DETAIL AREA SOURCE: R. LINEBACK

Palettes • The palette is the range of colors available for display • A palette is browser-safe if it can be displayed on all popular browsers • A browser-safe palette • Netscape color palette map

Netscape (Non-Dithering) Palette RGB (0%, 20%, 40%, 60%, 80%, 100%) 0% 20% 40% 60% 80% 100% = = = 0 51 102 153 204 255 = = = Hex Hex Hex 00 33 66 99 CC FF 6 choices for each of R, G, B = 216 colors SOURCE: WEBREFERENCE. COM

Palette of Indexed Colors • Color values indicate one of a set of an arbitrary set of colors, not RBG mixture. (LUT = look-up table) SOURCE: HANDBOOK OF GRAPHIC FILE FORMATS

Compression • • • There’s never enough space File (string) is a representation of information A representation (encoding) has a size Compression means reducing encoding size When can a string be compressed? – 00001000010000100001 – (00001)1000 means “ 00001” repeated 8 times – But some 40 -bit strings will get longer this way

Compression • An encoding is a “program” for generating the string • Strings having short programs can be compressed • Strings without short programs (Chaitin-Kolmogorov “random” strings) can’t be compressed • Compression limit: – n bits can only encode 2 n different strings • Dirichlet pigeonhole principle – If p+1 pigeons fly into p holes, some hole has more than one pigeon • Try to encode > 2 n strings in n bits loses information

Run-Length Encoding (RLE) • A run is a repeated character • Represent a run by two bytes: RUN-LENGTH CHARACTER • String “AAAAAAbbb. XXXXXt” (15 characters) become “ 6 A 3 b 5 X 1 t” (8 characters) • Advantages of RLE: fast, easy to code • What happens in RLE if no character is repeated?

3 -Byte Run-Length Encoding For repeated characters, use 3 bytes: “flag” + 2 - byte RLE. For solo characters, use 1 byte. SOURCE: HANDBOOK OF GRAPHIC FILE FORMATS

Bitmap (BMP) Format • Header • Bitmap data (indexed color or RGB) – 1, 4, 8, 16, 24, 32 - bit color • Palette • RLE, otherwise uncompressed • Generally very large

Dictionary Compression • Idea: choose a dictionary with < 65, 535 words • Number the words: – administration (3773), administrations (3774), fascinating (11012), xylophone (43722) • Every word can be represented by 2 bytes. (2 bytes = 16 bits = 0. . 65535) • Works for text • Doesn’t work for graphics

LZW (Lempel-Ziv-Welch) Compression • • • Start dictionary with all 1 -character strings C is next input character, P is empty Is P | C in the dictionary? If so, let P = P | C If not, – Output the code for P – Add P | C to the dictionary (and give it a code) – Set P = C • Repeat from “Is” until last character is seen, then output the code for P LZW Explanation Page (Mc. Gill)

LZW (Lempel-Ziv-Welch) Compression A B 1 2 B 3 A 4 B 5 A B 6 7 A C & 8 9 Dictionary = (1) A (2) B (3) C Step Pos 1 1 1 2 2 3 3 4 4 5 6 6 7 7 8 8 9 6 10 P A B B A AB ABA C C A B B A B A C Dict. Output (4) A B (5) B B (6) B A 1 2 2 (7) A B A 4 (8) A B A C 7 3 ALGORITHM: Is P | C in the dictionary? If so, let P = P | C If not, Output the code for P P | C dict (new code) Set P = C 1 2 2 4 7 3 = A B B A B A C! Dictionary not stored!

Graphic Image File (GIF) Format GIF identifier Screen dimensions Indexed color palette Position and size of image Indexed color palette LZW compressed data

GIF Compression • Since LZW is used, compression depends on randomness of image 941 BYTES 1004 BYTES 1452 BYTES 1803 BYTES 1239 BYTES 6685 BYTES SOURCE: KILLERSITES. COM

Compression Tradeoffs • Size v. Quality. bmp. gif. jpg 1000 KB, 172 KB, 47 KB, 13 KB, compression factor = 1 compression = 5. 8 high quality, compression = 21 low quality, compression = 75 • Size v. processing time – time to compress – time to decompress – not necessarily equal

Lossless v. Lossy Compression • A method of compression is “lossless” if no two strings become identical when compressed – RLE and LZW are lossless • Lossy compression: at least two strings compress to the same result – Example: Compress a 32 -bit string by taking just the first (leftmost) 16 bits (truncation) – If both A and B yield the compressed string C, we have “lost” the ability to recover the original • Lossy compression is OK for images if all possible decodings of C “look” almost the same

JPEG Format • JPEG = Joint Photographic Experts Group • Lossy compression • Discards visual info not seen by human eye (perceptual encoding) – slight changes in color not visible – slight changes in intensity easily visible – So, sacrifice color info in favor of intensity • Typical compression of 25: 1 without degradation • User can control quality

COMPRESSION RATIO 13. 5 SOURCE: NCSU

COMPRESSION RATIO 32. 4 SOURCE: NCSU

JPEG Compression CHANGE COLOR SPACE REDUCE COLOR INFO DISCRETE COSINE TRANSFORM DISCARD HIGH FREQUENCIES COMPRESS RESULT

JPEG Format • Operates on blocks of 8 x 8 pixels in the R, G and B planes • Must transform RGB into a color space that separates color from brightness:

Color Conversion and Downsampling • Must convert from RGB into a color space that separates color from brightness • Use YUV (human eye most sensitive to yellow) • Keep the Y (yellow luminance) data • Clip (shorten) the U, V data (U, V represent color differences so they are not very important)

Discrete Cosine Transform SOURCE: LULEÅ UNIV.

Discrete Cosine Transform

JPEQ Quantization (Truncation) • Idea: Threshold the values so that low ones (most of them) become zero:

Zig-Zag Run-Length Encoding • Zig-zag through the matrix so the tail of the sequence is all zeroes: 168, 45, 67, 12, 20, 7, 3, 3, 5, 2, 0, 0, 5, 3, 2, 0, 2, 2, 0, 0, . . . 64 numbers reduced to 18, a factor of ~ 4

JPEG Quality v. Compression One image is uncompressed (313 KB). One image is compressed by a factor of 13. 6 (23 KB). Which is which? SOURCE: E. DELP, PURDUE

JPEG Quality v. Compression NW: Quality = 75, Compression = 13. 6 NE: Q = 20, C = 37 SW: Q= 5, C = 78 SE: Q = 3, C= 96

JPEG Compression Example • Street scene from Simon Fraser University • Interactive JPEG example (Simon Fraser)

Sequential JPEG Image appears from top down:

Progressive JPEG Image “comes in” to focus:

Motion • Capture viewer – Requires concentration – Utilizes natural animal attention to movement • Involvement • Realism – Real-world object move • Convey message • Direct viewer’s attention

Delivering Video • Uncompressed video – 640 x 480 resolution – 24 -bit color = 7. 37 megabits/frame – 30 frames per second = 210 megabits/sec. – 1920 x 1080 = 1. 5 Gbps • Compressed video – MPEG-2. Broadcast quality. 15 Mbps – AVI, Quick. Time. 500 Kbps – Real. Networks. Designed for network speeds. 20 -200 Kbps. (At 20 Kbps, compression factor is greater than 10, 000. )

Streaming • Begin delivering content without full download – Lower latency – Low client storage requirement – Can start streaming at any point (not necessarily beginning – Live streaming possible (real-time compression) • Disadvantages – Lower quality, also affected by network connection – More time needed to compress – Need special server

1. True Streaming USER REQUESTS STREAMING MEDIA 2. 4. BROWSER REQUESTS pnm METAFILE BROWSER with video plug-in pnm METAFILE 5. CLIENT SIDE SERVER SIDE 3. SERVER UPLOADS METAFILE pnm METAFILE REALPLAYER PLUG-IN REQUESTS STREAM 6. WEB SERVER BROWSER INVOKES PLUG-IN REALSERVER UPLOADS STREAM REALSERVER STREAM

Q&A 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS