Intro ANN Fuzzy Systems Lecture 24 Radial Basis

Intro. ANN & Fuzzy Systems Outline • Interpolation Problem Formulation • Radial Basis Network

Intro. ANN & Fuzzy Systems What is Radial Basis Function? • RBF is a kernel function that is symmetric w. r. t. origin. Hence its variable is r that is the norm-distance from origin. • Examples of RBF (C) 2001 by Yu Hen Hu 3

Intro. ANN & Fuzzy Systems Interpolation Problem Formulation Radial Basis function for interpolation: Given {xi; 1 i K} and {di; 1 i K }, find a function F(x) that satisfies the interpolation condition: F(xi) = di 1 i K (1) One possible choice of F(x) is a radial basis function of the following form: (2) where {xi; 1 i K } are the centers of the radial basis functions. (C) 2001 by Yu Hen Hu 4

Intro. ANN & Fuzzy Systems Solving Radial Basis Coefficients • Substitute (1) into (2), we obtain a set of linear system of equations Mw=d (3) where M = [M(i, j), 1 i, j, K] is the interpolation matrix, M(i, j) = (||xi – xj||), w = [w 1, w 2, • • • , w. K]t, and d = [d 1, d 2, • • • , d. K]t. • Given M and d, assuming the N centers are distinct, w can be solved as: w = M 1 d if M is non-singular. If the (r) = (r 2 + c 2)– 1/2, or (r) = exp(–r 2/(2 s 2)), it can further be shown that M is also positive definite. (C) 2001 by Yu Hen Hu 5

Intro. ANN & Fuzzy Systems An Example • • Let F(– 1) = 0. 2, F(– 0. 5) = 0. 5, and F(1) = – 0. 5. Use a triangular radial basis function (r) = (1–r)[u(r) –u(r – 1)] u(r) = 1 if r 0 and = 0 if r < 0. (C) 2001 by Yu Hen Hu rbfexample 1. m 6

Intro. ANN & Fuzzy Systems Example continued Use Gaussian rbfs: Parzen window: No weighting,

Intro. ANN & Fuzzy Systems Example (Comparison) (C) 2001 by Yu Hen Hu 8

Intro. ANN & Fuzzy Systems Regularization Problem Formulation • When there are too many data points, the M matrix may become singular. • This is because by impose a rbf to each data point, we have an over-determined system. • Regularization is the mathematical tool that addresses this problem. (C) 2001 by Yu Hen Hu • By regularization, we add an additional term to the cost function that represents additional constraints on the solution: • Regularization term (e. g. ): 9

Intro. ANN & Fuzzy Systems Solution to Regularization Problem • The solution to this regularization problem is • Hence • G(x; xi) is the Green's function corresponding to the selfadjoin differential operator P*P such that P*P G(x; xi) = d(x – xi) • A solution to the Green function that is of special interests to us is a multi-variate Gaussian function (C) 2001 by Yu Hen Hu • With individual training data substituted into G(x, xi), a matrix equation (G + l. I) w = d can be solved for w. 10

Intro. ANN & Fuzzy Systems Implementation Consideration • However, other radial basis function other than the multi-variate Gaussian rbf can also be used. • The regularized F(x) may no longer match data points exactly, but it will be more smooth. • The value of l is usually determined empirically although generalized cross -validation (GCV) may be applied here. (C) 2001 by Yu Hen Hu 11