Basis Basics Selected from presentations by Jim Ramsay

Basis Basics Selected from presentations by Jim Ramsay, Mc. Gill University, Hongliang Fei, and Brian Quanz

1. Introduction n Ø Ø Basis: In Linear Algebra, a basis is a set of vectors satisfying: Linear combination of the basis can represent every vector in a given vector space; No element of the set can be represented as a linear combination of the others.

n n n In Function Space, Basis is degenerated to a set of basis functions; Each function in the function space can be represented as a linear combination of the basis functions. Example: Quadratic Polynomial bases {1, t, t^2}

What are basis functions? n n n We need flexible method for constructing a function f(t) that can track local curvature. We pick a system of K basis functions φk(t), and call this the basis for f(t). We express f(t) as a weighted sum of these basis functions: f(t) = a 1φ1(t) + a 2φ2(t) + … + a. KφK(t) The coefficients a 1, … , a. K determine the shape of the function.

What do we want from basis functions? n n n Fast computation of individual basis functions. Flexible: can exhibit the required curvature where needed, but also be nearly linear when appropriate. Fast computation of coefficients ak: possible if matrices of values are diagonal, banded or sparse. Differentiable as required: We make lots of use of derivatives in functional data analysis. Constrained as required, such as periodicity, positivity, monotonicity, asymptotes and etc.

What are some commonly used basis functions? n n n Powers: 1, t, t 2, and so on. They are the basis functions for polynomials. These are not very flexible, and are used only for simple problems. Fourier series: 1, sin(ωt), cos(ωt), sin(2ωt), cos(2ωt), and so on for a fixed known frequency ω. These are used for periodic functions. Spline functions: These have now more or less replaced polynomials for non-periodic problems. More explanation follows.

What is Basis Expansion? n Given data X and transformation Then we model as a linear basis expansion in X, where is a basis function.

Why Basis Expansion? n n n In regression problems, f(X) will typically nonlinear in X; Linear model is convenient and easy to interpret; When sample size is very small but attribute size is very large, linear model is all what we can do to avoid over fitting.

2. Piecewise Polynomials and Splines n Ø Ø n Spline: In Mathematics, a spline is a special function defined piecewise by polynomials; In Computer Science, the term spline more frequently refers to a piecewise polynomial (parametric) curve. Simple construction, ease and accuracy of evaluation, capacity to approximate complex shapes through curve fitting and interactive curve design.

n Assume four knots spline (two boundary knots and two interior knots), also X is one dimensional. Piecewise constant basis: n Piecewise Linear Basis: n

Piecewise Cubic Polynomial n n Basis functions: Six functions corresponding to a sixdimensional linear space.

Piecewise Cubic Polynomial

Spline Interpolation Method Slides taken from the lecture by Authors: Autar Kaw, Jai Paul http: //numericalmethods. eng. usf. edu 14

What is Interpolation ? Given (x 0, y 0), (x 1, y 1), …… (xn, yn), find the value of ‘y’ at a value of ‘x’ that is not given. 15 lmethods. eng. usf. edu http: //numerica

Interpolants Polynomials are the most common choice of interpolants because they are easy to: Evaluate Differentiate, and Integrate. 16 lmethods. eng. usf. edu http: //numerica

Why Splines ? 17 lmethods. eng. usf. edu http: //numerica

Why Splines ? 18 Figure : Higher order polynomial interpolation is a bad ideahttp: //numerica lmethods. eng. usf. edu

Linear Interpolation 19 lmethods. eng. usf. edu http: //numerica

Linear Interpolation (contd) 20 lmethods. eng. usf. edu http: //numerica

Example The upward velocity of a rocket is given as a function of time in Table 1. Find the velocity at t=16 seconds using linear splines. Table Velocity as a function of time (s) 0 10 15 20 22. 5 30 21 (m/s) 0 227. 04 362. 78 517. 35 602. 97 901. 67 Figure. Velocity vs. time data for the rocket example lmethods. eng. usf. edu http: //numerica

Linear Interpolation 22 lmethods. eng. usf. edu http: //numerica

Quadratic Interpolation 23 lmethods. eng. usf. edu http: //numerica

Quadratic Interpolation (contd) 24 lmethods. eng. usf. edu http: //numerica

Quadratic Splines (contd) 25 lmethods. eng. usf. edu http: //numerica

Quadratic Splines (contd) 26 lmethods. eng. usf. edu http: //numerica

Quadratic Splines (contd) 27 lmethods. eng. usf. edu http: //numerica

Quadratic Spline Example The upward velocity of a rocket is given as a function of time. Using quadratic splines a) Find the velocity at t=16 seconds b) Find the acceleration at t=16 seconds c) Find the distance covered between t=11 and t=16 seconds Table Velocity as a function of time (s) 0 10 15 20 22. 5 30 28 (m/s) 0 227. 04 362. 78 517. 35 602. 97 901. 67 Figure. Velocity vs. time data for the rocket example lmethods. eng. usf. edu http: //numerica

Solution Let us set up the equations 29 lmethods. eng. usf. edu http: //numerica

Each Spline Goes Through Two Consecutive Data Points 30 lmethods. eng. usf. edu http: //numerica

Each Spline Goes Through Two Consecutive Data Points 31 t s 0 10 v(t) m/s 0 227. 04 15 20 22. 5 30 362. 78 517. 35 602. 97 901. 67 lmethods. eng. usf. edu http: //numerica

Derivatives are Continuous at Interior Data Points 32 lmethods. eng. usf. edu http: //numerica

Derivatives are continuous at Interior Data Points At t=10 At t=15 At t=20 At t=22. 5 33 lmethods. eng. usf. edu http: //numerica

Last Equation 34 lmethods. eng. usf. edu http: //numerica

Final Set of Equations 35 lmethods. eng. usf. edu http: //numerica

Coefficients of Spline 36 i ai bi ci 1 0 22. 704 0 2 0. 8888 4. 928 88. 88 3 − 0. 1356 35. 66 − 141. 61 4 1. 6048 5 0. 20889 − 33. 956 554. 55 28. 86 − 152. 13 lmethods. eng. usf. edu http: //numerica

Quadratic Spline Interpolation Part 2 of 2 http: //numericalmethods. eng. usf. edu 37 lmethods. eng. usf. edu http: //numerica

Final Solution 38 lmethods. eng. usf. edu http: //numerica

Velocity at a Particular Point a) Velocity at t=16 39 lmethods. eng. usf. edu http: //numerica

Quadratic Spline Graph t=a: 2: b;

Quadratic Spline Graph t=a: 0. 5: b;

Natural Cubic Spline Interpolation SPLINE OF DEGREE k = 3 n n n The domain of S is an interval [a, b]. S, S’’ are all continuous functions on [a, b]. There are points ti (the knots of S) such that a = t 0 < t 1 <. . tn = b and such that S is a polynomial of degree at most k on each subinterval [t i, ti+1]. x t 0 t 1 … tn y y 0 y 1 … yn ti are knots

Natural Cubic Spline Interpolation Si(x) is a cubic polynomial that will be used on the subinterval [ xi, xi+1 ].

Natural Cubic Spline Interpolation n Si(x) = aix 3 + bix 2 + cix + di • • 4 Coefficients with n subintervals = 4 n equations There are 4 n-2 conditions • Interpolation conditions Continuity conditions • Natural Conditions • • • S’’(x 0) = 0 S’’(xn) = 0