Audio Video Representation CS 105 Data Representation Types

Audio & Video Representation CS 105

Data Representation • Types of data: – Numbers – Text – Images – Audio & Video

What is sound? • A continuous wave created by oscillations of pressure through any material (solid, liquid & gas) Analog signal • For a computer: Analog to Digital – Specialized Hardware

Digitizing Sound Quantization Voltage Sampling Time • Discrete time (sampling) & discrete voltage (quantization) • Reasonable sound production: 40, 000 times per sec • Quantization: Dividing vertical axis into pieces – process of mapping a continuous range of values by a small finite set of values • 8 bit quantization = 256 levels, 16 bit = 65536 levels

Digitizing Sound Quantization into levels Sampling time intervals How many bits are needed to represent per sample? - 3 bits - Power of 2 How many bits are needed to digitize this signal? - 3 x 13 = 39 bits

Bit Depth and Bit Rate • Bit Depth: Number of bits per sample • Bit Rate: Number of bits that are sent per unit of time q Each tick on the time axis (x-axis) is 1 sec. All levels are represented by 3 bits. What is the bit rate? - 3 bits/s 1 sec q Every 4 ticks on the time axis is 1 sec i. e. you sample 4 times a second. What is the bit rate now? - 12 bits/s

Audio Formats • Wav, AU, AIFF, VQF and MP 3 • MP 3 (MPEG 1 or 2) – dominant format for compressing audio data – Psychoacoustic models – Form of Huffman encoding – Lossy compression • Bit rates: – Uncompressed audio: 1411 kbit/s – Compressed audio (such as mp 3) : 128 to 320 kbit/s

Practice Digitizing Signals!

Name: Date: Voltage Level 4 Level 3 Level 2 Level 1 S 2 S 3 S 4 S 5 S 6 Digitize this graph in the sampling intervals using 2 bits Time

Video • Three dimensional array of color pixels – Succession of images – Each image in the video is called a frame • Video: Duration, Frame Size with a bit depth, frame rate (fps) – Example video: Duration: 3600 s (1 hr), Frame Size: 640 W*480 H with 24 bits for each pixel, 25 fps • Frame: 0. 0009216 Gb, Every sec: 0. 023 Gb, • Every hour: ~83 Gb – Human eye and frame rate: film and video games

Video Codecs • Video files are big! – Typically gigabytes of data for an hour of video • Codec: Compressor/Decompressor – Employs lossy compression – Temporal & Spatial compression • Temporal compression: Differences between frames – Keyframe is used to compare differences between frames and encode only the differences • Spatial compression: Similar regions within frames and groups them – Run length encoding

Container formats • Encapsulates both audio, video and metadata (needed to synchronize audio and video streams) • Stores the data, does not encode it – For access: program that opens the container and decodes the data inside • AVI, MOV

Video Data • MPEG 1, 2 and 4 lossy compression – MPEG 1 and 2 – deals with differences in frames – MPEG 4 : • Better video quality than MPEG 1 and 2 • Widely used • Instead of dealing with each individual frame, it tackles objects within each frame • Bit rate: – 1. 25 Mbit/s – VCD quality (MPEG-1 video compression) – 5 Mbit/s – DVD quality (MPEG-2 compression) – 8 to 15 Mbit/s – HDTV quality (MPEG-4 compression) – 40 Mbit/s max – Blu-ray Disc