SAMPLING QUANTIZATION Dhany Arifianto How might you measure

SAMPLING & QUANTIZATION Dhany Arifianto

How might you measure waves on a beach? Or, even, sample the wave height

Is sampling okay? Question: Is a sampled representation the same thing? – Yes, provided…

• Digital means discrete (like whole numbers) and Analog means continuous (like physical properties such as temperature, volume, etc. ). • The term analog comes from early computers (circa WWII) used to solve differential equations with continuous variables, • as contrasted with discrete state machines (like an elevator controller) built from openor-closed switches or on-off digital circuits

Definitions (from whatis. com) Analog • Using physical representation • Relating to a system, device that represents data variation by a measurable physical quality such as temperature, volume, distance, weight, pressure … • Which is continuous in time or space and value

Definitions Digital • Representing data as numbers – – – Processing Operating on Storing Transmitting Displaying • Data in the form of numerical digits, as in a digital computer

• Representing a physical quantity – such as sound, light, or electricity • by means of samples – taken at discrete times (or places) – and given numerical values • usually in the binary system – as in a digital audio recording – or in digital television – or in digital photography

Analog vs. Digital • Analog a In them ma se sen – Media represented using real values – Sound pressure in the air (example) – Electronic representation of sound from a microphone • Digital – Media represented using discrete values • (integers, quantized numbers, floating point representations) • May be infinite, but for a fixed range finite. l ca i t a

Electronic capture of sounds Analog pressure variations N S Analog voltage variations

In Communications • Analog is used to refer to systems with signals that are continuous in value and time – such as AM and FM, where the electrical signals are representations of the information signals.

A N A L O G Amplitude Modulation (AM)

Phase or Frequency Modulation (FM)

In Communications • Digital is used to refer to discrete-state, discrete-time signals that can take on only specific values at specific times; • such as – sampled/quantized signals, – pulse modulated signals, • and to data communication signals in general.

Digital Modulation: Discrete in Time and Value

Parameters of Information Sources & Systems • Analog (continuous functions of time, space, weight, …) – voice, audio, image, video, temperature • Bandwidth – frequency (harmonics) range • Statistics – amplitude distribution, power, spectrum (frequency content, harmonics) • Digital (sets of numbers): – ASCII characters, computer words, … • Bit Rate – bps, kbps, Mbps, Gbps, Tbps, Ebps, …

How does Information Become “Digital”?

Digital Representation • Information that is naturally discrete, such as state of a light switch (on-off), integers, or text can be represented by binary numbers in obvious ways. • Text (as generated on a keyboard) is often represented by 8 -bit binary numbers. • Speech may be represented by a pressure wave, which is continuous – in time and value – and has to be sampled and quantized to be represented digitally.

Discrete Information • Some information, such as numerals and characters is discrete and can be represented “digitally” easily • Take characters of the English Language for example • The American Standard Code for Information Interchange (ASCII) is the binary representation used in teletype messaging and adopted as a universal computer character representation.

“A” = 11000001 “a” = 11100001 “%” = 10100101 Formatting 10001101 = CR 10001010 = LF Messaging 10000001 = SOH 10000010 = STX 10000011 = ETX 10000100 = EOT

Common Sense Digitization of Analog Information • All continuous signals can be represented by a collection of numbers to any degree of accuracy by – sampling often enough and – using enough quantization levels* to represent the signal value at the sampling instants. – * determined by the number of digits in the representation

Analog-to-Digital Conversion • • Two stage process Sample – Sampling Theorem – Nyquist Rate • Quantize – Precision, SNR (% average error) – Note: a digital representation of an analog value always has error

The Sampling Theorem • Shannon’s Sampling Theorem states that – any bandlimited signal may be represented by samples taken at a rate of twice its highest frequency*, and – may be reconstructed without error if the appropriate interpolation functions are used**. * Twice the highest frequency is called the Nyquist Rate. ** Physically unrealizable sinx/x or (sinc) functions. This theorem is also known as Nyquist–Shannon–Kotelnikov Whittaker–Nyquist–Kotelnikov–Shannon WKS Cardinal Theorem of Interpolation Theory

The Sampling Theorem Harry Nyquist, working at Bell Labs developed what has become known as the Nyquist Reconstruction Theorem: Given a uniform sample rate of fs, the highest frequency that can be unambiguously represented is fs/2 – fs/2 is the Nyquist Frequency CD Players use fs=44, 100 samples/sec – The Nyquist Frequency is 22, 050 Hz – Why don’t CD’s sample at 40 KHz?

Sampling Lets consider a sign wave to be sampled (e. g. voice signal) If you sample at 1 time per cycle, you can think it is a constant Sampling 1. 5 times each cycle appears as a low frequency sine signal Nyquist Theorem: For Lossless digitization, the sampling rate should be at least twice the maximum frequency responses

Representation of a CT Signal Using Impulse Functions • The goal of this lecture is to convince you that bandlimited CT signals, when sampled properly, can be represented as discrete-time signals with NO loss of information. This remarkable result is known as the Sampling Theorem. x(t) • Recall our expression for a pulse train: … … t -2 T -T 0 T • A sampled version of a CT signal, x(t), is: This is known as idealized sampling. • We can derive the complex Fourier series of a pulse train: 2 T

Sampling simply taking readings at fixed points in time Uniform sampling – sampling at regular intervals

Impulse Sampling

Fourier Transform of a Sampled Signal • The Fourier series of our sampled signal, xs(t) is: • Recalling the Fourier transform properties of linearity (the transform of a sum is the sum of the transforms) and modulation (multiplication by a complex exponential produces a shift in the frequency domain), we can write an expression for the Fourier transform of our sampled signal: • If our original signal, x(t), is bandlimited:

Sampling of Narrowband Signals • What is the lowest sample frequency we can use for the narrowband signal shown to the right? • Recalling that the process of sampling shifts the spectrum of the signal, we can derive a generalization of the Sampling Theorem in terms of the physical bandwidth occupied by the signal. • A general guideline is , where B = B 2 – B 1. • A more rigorous equation depends on B 1 and B 2: • Sampling can also be thought of as a modulation operation, since it shifts a signal’s spectrum in frequency.

Signal Reconstruction • Note that if , the replicas of do not overlap in the frequency domain. We can recover the original signal exactly. • The sampling frequency, sampling frequency. , is referred to as the Nyquist • There are two practical problems associated with this approach: § The lowpass filter is not physically realizable. Why? § The input signal is typically not bandlimited. Explain.

Signal Interpolation • The frequency response of the lowpass, or interpolation, filter is: • The impulse response of this filter is given by: • The output of the interpolating filter is given by the convolution integral: • Using the sifting property of the impulse:

Signal Interpolation (Cont. ) • Inserting our expression for the impulse response: • This has an interesting graphical interpretation shown to the right. • This formula describes a way to perfectly reconstruct a signal from its samples. • Applications include digital to analog conversion, and changing the sample frequency (or period) from one value to another, a process we call resampling (up/down). • But remember that this is still a noncausal system so in practical systems we must approximate this equation. Such implementations are studied more extensively in an introductory DSP class.

Reconstruction

Sampling Analog signal Sample and Hold Sampled signal Real values at fixed points in time Uses a “sample-and-hold” Values are called “samples” Sample Analog Devices AD 585 Sample and Hold Amplifier

Sound in the Real World Recall: All sounds can be expressed as a combination of sinusoids String instruments generate harmonics C 8 on piano = 4186 Hz 6 th Harmonic is 25, 116 Hz Above the Nyquist Frequency!! Many instruments generate content above 20 KHz

A short section of a speech waveform (highest frequency component is 3 KHz) Reconstructed speech waveform with 1 KHz sampling rate (note the resulting waveform does not resemble original waveform)

Undersampling & Oversampling n n Undersampling n Sampling at an inadequate frequency rate n Aliased into new form - Aliasing n Loses information in the original signal Oversampling n Sampling at a rate higher than minimum rate n More values to digitize and process n Increases the amount of storage and transmission n COST $$

Aliasing • Recall that a time-limited signal cannot be bandlimited. Since all signals are more or less time-limited, they cannot be bandlimited. Therefore, we must lowpass filter most signals before sampling. This is called an anti-aliasing filter and are typically built into an analog to digital (A/D) converter. • If the signal is not bandlimited distortion will occur when the signal is sampled. We refer to this distortion as aliasing: • How was the sample frequency for CDs and MP 3 s selected?

Antialiasing Filters We have to filter (remove) any content above the Nyquist Frequency Analog signal Antialiasing Filter Sample and Hold Not digital devices!!! Sampled signal

Analog filtering • Filtering is the relative modification of amplitudes of different frequencies Flat response High (treble) boost Low (bass) cut 2 1 0 0 10 k 20 k 30 k

Antialiasing filters • What we want is a brick wall filter, that passes everything below 20 KHz and cuts everything above 2 1 0 0 10 k 20 k 30 k

Reality vs. Ideal • But, we can’t build a brick wall filter We can specify: Maximum ripple Pass band Cut band Minimum cut Transition band

Real analog filters • We pay for transition/pass ratio Costs: Component count Drift Noise Non-linearity Phase distortion Other junk… pass tr

Undersampling and Oversampling of a Signal

Sampling is a Universal Engineering Concept • Note that the concept of sampling is applied to many electronic systems: § electronics: CD players, switched capacitor filters, power systems § biological systems: EKG, EEG, blood pressure § information systems: the stock market. • Sampling can be applied in space (e. g. , images) as well as time, as shown to the right. • Full-motion video signals are sampled spatially (e. g. , 1280 x 1024 pixels at 100 pixels/inch) , temporally (e. g. , 30 frames/sec), and with respect to color (e. g. , RGB at 8 bits/color). How were these settings arrived at?

Downsampling and Upsampling • Simple sample rate conversions, such as converting from 16 k. Hz to 8 k. Hz, can be achieved using digital filters and zero-stuffing:

Oversampling • Sampling and digital signal processing can be combined to create higher performance samplers • For example, CD players use an oversampling approach that involves sampling the signal at a very high rate and then downsampling it to avoid the need to build high precision converter and filters.

Summary • All signals can be represented by a collection of numbers to any degree of accuracy by sampling often enough and using enough quantization levels to represent the signal value at the sampling instant.

Summary • Shannon’s Sampling Theorem states that any strictly bandlimited function may be presented by sampling at a rate that is at least twice as fast as the highest frequency in the signal, and that it may be recovered without distortion by passing the (impulse) samples through an ideal low-pass filter with a bandwidth equal to that of the signal.

Quantization • For processing, storage or communication, samples with infinite precision must be quantized • Such that a range, or interval, of values is represented by a single, finite precision, number • For example, by a finite binary number.

Quantization 7 7 7 6 5 5 4 3 3 2 1 -3 2 1 1 -2 -3 time -2 -3 -4

Reconstitution 7 7 7 6 5 5 4 3 3 2 1 1 -3 2 1 -2 -3 time -2 -3 -4 -2 Quantum Boundary Actual Value Reconstruction Value -3 ERROR -3 Quantum Boundary -4

Quantization (linear vs. logarithmic) Logarithmic Quantization Linear Quantization The 127 quantization levels are spread evenly over the voice signal’s dynamic range u This gives loud voice signals the same degree of resolution (same step size) as soft voice signals u Encoding an analog signal in this manner, while conceptually simplistic, does not give optimized fidelity in the reconstruction of human voice u Most of the energy in human voice is concentrated in the lower end of voice’s dynamic range (no shouting – just from boss) u Quantization levels distributed according to a logarithmic, instead of linear, function gives finer resolution, or smaller quantization steps, at lower signal amplitudes u PCM in North America uses a logarithmic function called μ-law u

Quantization Error (for nerds and audiophiles) • The quantization error depends on the number of distinct quantization intervals used. • If N binary digits are used, the number of distinct intervals is 2 N. • The signal-to-quantization-error ratio is about (6 N + 1. 8) d. B.

Binary Representation • Once information is discretized, or sampled, a number can be assigned to represent the value of each sample. • The number can be expressed as a binary number, e. g. , 2009 is 1024 + 512 + 256 + 128 +64 + 32 + 8 + 4 + 1 1 x 210 + 1 x 29 + 1 x 28 + 1 x 27 + 1 x 26 +1 x 25 +1 x 23 + 1 x 22 + 1 x 20 11111101101

Example: CD Audio Human hearing band: 20 -20000 Hz Passband: 0 -20000 Hz Transition band: 0. 10 * 20000 = 2000 10% transition band Band is 20000 -22000 Hz Minimum possible sample rate: 2 * 22000 = 44000 samples/second They actually use 44, 100 samples/second Any ideas why? Example: Professional Audio Human hearing band: 20 -20000 Hz Passband: 0 -20000 Hz Transition band: 0. 20 * 20000 = 4000 20% transition band Band is 20000 -24000 Hz Minimum possible sample rate: 2 * 24000 = 48000 samples/second Most modern equipment has moved to 96, 000 samples/second a passband of 0 -40000 Hz

Why 44, 100 for CDs? 44, 100 is evenly divisible by: – – 60 (American/Japanese television field rate) 50 (European television field rate) 30 (American/Japanese television frame rate) 25 (European television frame rate) The exact frame rate in the US/Japan is actually 29. 97, which is not divisible, but there are lots of funky numbers in the television signal…

Choosing a sample rate for analog sampling 1. 2. 3. 4. Select a passband Choose a transition band ratio Design an antialiasing filter Select a sample rate such that fs/2 is above the transition band Generally you’ll use standard sample rates Remember, each sample is a data point ! Those rules are applicable in DAC/ADC design(for signal processing, control, etc. )

Converting Digital to Analog Digital Signal Assume: Unsigned 8 bit value, analog range 1 volt Analog 1/2 V Voltage Adder 1/4 V 1/8 V 1/16 V 1/32 V 1/64 V 1/128 V 1/256 V

Converting Analog to Digital Binary search for correct value a a>b? Result Successive Approximation (Binary search) D/A Conversion b