Orthogonal Transforms Fourier Cosine KL Walsh Hadamard Orthogonal

Orthogonal Transforms Fourier Cosine KL Walsh Hadamard

Orthogonal Functions

Mean Square Error

Important result

• We want to minimize this kinds of errors. • Other error measures are also used.

Example of arbitrary basis functions being rectangular waves

This determining first function determines next functions

0 1

Small error with just 3 coefficients

This slide shows four base functions multiplied by their respective coefficients

This slide shows that using only four base functions the approximation is quite good End of example

General Orthogonal Transforms

Extending general transforms to 2 dimensions

Orthogonal and separable Image Transforms Forward transform inverse transform separable

Unitary Transforms • Unitary Transformation for 1 -Dim. Sequence – Series representation of – Basis vectors : – Energy conservation : Here is the proof

• Unitary Transformation for 2 -Dim. Sequence – Definition : – Basis images : – Orthonormality and completeness properties • Orthonormality : • Completeness :

• Unitary Transformation for 2 -Dim. Sequence – Separable Unitary Transforms • separable transform reduces the number of multiplications and additions from to – Energy conservation

Fourier Transform separable

Extension of 1 -D Fourier Transform to two dimensions

Discrete Cosine Transform (DCT)

Discrete Cosine Transform (DCT)

Discrete Cosine Transform (Compare to WH transform)

DCT is an orthogonal transformm so its inverse kernel is the same as forward kernel

2 -D Discrete Cosine Transform

2 -D Discrete Cosine Transform

Properties of DCT: real, orthogonal, energycompacting, eigenvector-based

– Basis Functions for 1 -dim. DCT(N = 16)

There are many DCT fast algorithms and hardware designs. Many fast algorithms are available fast algorithm : Lee(1 -D), Lee-Cho(2 -D) VLSI algorithm : regularity, local interconnection, moduality ref: 1. Nam Ik Cho and Sang Uk Lee, “Fast algorithm and implementation of 2 -D DCT. ” IEEE Trans Circuits and Systems, vol. 38, no. 3, pp. 297 -305, March 1991. 2. Nam Ik Cho and Sang Uk Lee, “DCT algorithm for VLSI parallel implementation” IEEE Trans. ASSP, vol. ASSP-38, no. 1, pp. 121 -127, Jan, 1990.

Discrete Sine Transform(DST) Similar to DCT.

• 1 -dim. DST (cont. ) – Basis Functions for 1 -dim. DST (N=16)

Karhunen-Leove (KL) Transform (Principal Component Analisis, PCA) Covariance matrix

• Methodology − Suppose x 1, x 2, . . . , x. M are N x 1 vectors

• Methodology – cont.

• Linear transformation implied by KL transform − The linear transformation RN RK that performs the dimensionality reduction is: 46

Principal Component Analysis (PCA) • Geometric interpretation − PCA projects the data along the directions where the data varies the most. − These directions are determined by the eigenvectors of the covariance matrix corresponding to the largest eigenvalues. − The magnitude of the eigenvalues corresponds to the variance of the data along the eigenvector directions. 47

• Eigenvalue spectrum K λi λN 48

Principal Component Analysis (PCA) • How many principal components to keep? − To choose K, you can use the following criterion (total variations preserved) : 49

• Representing faces onto this basis 50

Walsh Transform

Here we calculate the matrix of Walsh coefficients

Here we calculate the matrix of Walsh coefficients

Here we calculate the matrix of Walsh coefficients

We have done it earlier in different ways Here we calculate the matrix of Walsh coefficients

Symmetry of Walsh Think about other transforms that you know, are they symmetric?

Two-Dimensional Walsh Transform

Two-dimensional Walsh Inverse Two-dimensional Walsh

Properties of Walsh Transforms

Here is the separable 2 -Dim Inverse Walsh

Example for N=4

odd even

Discuss the importance of this figure

Hadamard Transform We will go quickly through this material since it is very similar to Walsh

separable

Example of calculating Hadamard coefficients – analogous to what was before

Standard Trivial Functions for Hadamard One change two changes

Discrete Walsh-Hadamard transform (Summary) Now we meet our old friend in a new light again!

Haar Transform • Haar transform – Haar function (1910, Haar) : periodic, orthonormal, complete Nonsinusoidal orthogonal function

Slant transform