Lecture 11 Representing Multimedia Digitally Digitizing Data Text

Lecture 11 Representing Multimedia Digitally

Digitizing Data • Text is easy • What about multimedia? – Photos, audio, and video • Same principles

Color and the Mystery of Light • Color image – Grid of pixels • Pixel is formed from three primary colors – RGB

Showing Colors • Colors formed by using 3 intensities of primaries – Full intensity red, green, or blue – Full intensity of red, green, and blue? – No intensity of any color? – Other combinations

LCD Display Technology • Close-up of white arrow pointer – Note subpixels

Black and White Colors • Intensity of light color stored in byte • Needs 3 bytes/pixel – Smallest intensity is 0000 • Decimal? – Largest value is 1111 • Decimal?

Black and White Colors • Black is absence of light: – 0000 0000 • White is full intensity of each color: – 1111 1111

Color Intensities • Consider blue (0000 • 8 bits have position values 1111) 128 64 32 16 8 4 2 1 1 1 1 1 • To cut intensity in half 128 64 32 16 8 4 2 1 1 0 0 0 0

Color Intensities

Decimal to Binary • How to convert decimal to binary? – Look for powers of 2 and subtract • E. g. : Convert 365 to binary

Lighten Up: Changing Color by Addition • What color does this represent? 1100 1000 • It’s RGB (200, 200), a grey – #C 8 C 8 in hex – All grays of form (x, x, x)

To Increase Intensity: Add in Binary • Increase common value to lighten – E. g. , add 0001 0000 (decimal 16) to each color 1101 1000 RGB (216, 216)

Lightening Adding another 16… 1101 1000 + 0001 0000 -------- Check using decimal!

Lighter Still: Adding with Carry Digits • Binary addition is similar to decimal addition – Work from right to left – 0 + 0 = 0, 0 + 1 = 1, 1 + 0 = 1 • No carry – 1 + 1 = 0 • Carry of 1

Binary Addition

Binary Addition

Binary Addition

Computing on Representations • Example: changing the brightness and contrast of a photo

Brightness and Contrast • Brightness – How close to white • Contrast – Difference b/w darkest and lightest portions of image • Photo manipulation software often gives values of pixels in a Levels graph

Levels Graph • 0 percent is black point (0, 0, 0) • 100 percent is white point (ff, ff) • Midpoint of pixel range is gamma point

Brightness • Shift pixels closer to white • Add 16 to each pixel • E. g. : (197, 197) => (213, 213)

Contrast • Scale pixel range – Stretch toward right • Add to each pixel, but – add a smaller amount for dark pixels – add a larger amount for light pixels

New Levels Graph

Adding Color • Color => (x, y, z), all 3 differ • Example – Colorize image of moon

Making the Moon Orange • Tint white regions – Pick a shade of orange, say (255, 213, 132) • Tint light gray – Red byte: leave unchanged – Green byte: reduce green slightly (subtract 42) – Blue byte: reduce blue significantly (subtract 123)

Digitizing Sound • Vibrating object creates sound • Vibrations “push” air to form pressure wave • Wave vibrates our eardrums

Digitizing Sound • Intensity of push determines volume • Frequency (# of waves per second) of pushes determines pitch continuous (analog) representation of the wave

Analog to Digital • Need to digitize to bits • Use binary # for amplitude of wave • At what point do you measure? – Infinitely many possible

Analog to Digital • Sample at regular intervals • Samples/second is sampling rate

Nyquist Rule for Sampling • Sampling rate is key • Nyquist rule – Sampling rate must 2 x highest frequency – Range of human hearing 20 Hz – 20 KHz • Digital audio sampling rate is 44. 1 KHz

Digitizing Process

Digitizing Process • Recording (digitizing) process – Sound => mic – Signal sampled by ADC • Samples encoded in binary

Digitizing Process • Playing process – Numbers read by DAC – Electrical wave created by interpolation – Electrical signal => speaker

How Many Bits per Sample? • Perfect accuracy requires unlimited bits/sample • Must handle both +/- values • More bits => more accuracy

How Many Bits per Sample? • More bits yields a more accurate digitization • Digital audio uses 16 bits

Advantages of Digital Sound • Key advantage is ability to compute on representation – Remove noise – Compression • Lossless • Lossy – Frequencies outside our range

Advantages of Digital Sound • MP 3 format – Allows for compression ratio > 10: 1 • Another key advantage of digital representations is exact reproduction

Digital Images and Video • Image is grid of RGB pixels • Stored as linear sequence • Can take up a lot of space

Digital Images and Video •

Image Compression • Typical monitor has fewer than 100 ppi – 9 x space saving over 300 ppi • Still requires more than 5. 5 min to send

Image Compression • Compression changes rep to use fewer bits – Example: faxes • Faxes are a sequences of 0’s and 1’s • Use run-length encoding

Compression • Run-length encoding is “lossless” • Opposite is “lossy compression”

Compression • MP-3 – Lossy scheme – Highs and lows lost • JPG (or JPEG) – Lossy scheme for images – Exploits limits of human perception • Luminance sensitivity • Chrominance insensitivity

JPEG Compression • JPEG is capable of 10: 1 compression w/o detectable loss of clarity

JPEG Compression • Ratios higher than 10: 1 – Smaller files – Possible artifacts (pixelation)

MPEG Compression • MPEG used for video • Video is sequence of stills – Each image/frame is not seen for long

MPEG Compression • JPEG used to compress frames • “Interframe coherency” is used – MPEG compression only transmits “deltas” b/w frames – Results in significant compression

Optical Character Recognition • OCR – Converts images of characters to characters – Used by U. S. P. S. – Used in banking – Used in automatic license-plate recognition

Virtual Reality • VR – Simulates real-world environment – Digitizes 3 D space, sound – Uses head-mounted displays – Training: surgeons, military

Haptic Devices • Haptic devices – Used in VR – Input/output technology for sense of touch – Gloves can apply forces

Latency • Challenges with VR – Latency • Time it takes for info to be recv’d – Bandwidth • Info per unit time – Fundamental limit: speed of light

Bits Are It! • 4 bytes can represent different info – Number – 4 characters – Color – etc. • Bias-Free Universal Medium Principle – Bits can represent all discrete info – Bits have no inherent meaning

Bits: Bias-Free • What does this bit sequence represent? 0000 1111 0000 1000 0010 0000 In hex: 00 F 1 08 20 • Depends on context

Bits are bits…

Summary • RGB color • Manipulating images • Binary numbers and arithmetic • Digitizing sound, images, video – Compression

Summary • OCR • Virtual Reality • Latency and Bandwidth • Bias-Free Universal Medium Principle

Quiz • How many colors can 9 bits represent? • 1010 + 0110 = ? • Convert C 7 to binary