Basis Basics Jim Ramsay Mc Gill University Overview

Basis Basics Jim Ramsay Mc. Gill University

Overview § What are basis functions? § What properties should they have? § How are they usually constructed? § Fourier bases. § B-spline bases: Lots of detail here. § Wavelet bases. § Advice, wrap-up, and beyond.

What are basis functions? § We need flexible method for constructing a function f(t) that can track local curvature. § We pick a system of K basis functions φk(t), and call this the basis for f(t). § We express f(t) as a weighted sum of these basis functions: f(t) = a 1φ1(t) + a 2φ2(t) + … + a. KφK(t) The coefficients a 1, … , a. K determine the shape of the function.

What do we want from basis functions? § Fast computation of individual basis functions. § Flexible: can exhibit the required curvature § § § where needed, but also be nearly linear when appropriate. Fast computation of coefficients ak: possible if matrices of values are diagonal, banded or sparse. Differentiable as required: We make lots of use of derivatives in functional data analysis. Constrained as required, such as periodicity, positivity, monotonicity, asymptotes and etc.

What are some commonly used basis functions? § Powers: 1, t, t 2, and so on. They are the basis functions for polynomials. These are not very flexible, and are used only for simple problems. § Fourier series: 1, sin(ωt), cos(ωt), sin(2ωt), cos(2ωt), and so on for a fixed known frequency ω. These are used for periodic functions. § B-spline functions: These have now more or less replaced polynomials for non-periodic problems. More explanation follows.

How do we construct basis systems? § We start with a prototype basis function φ(t). Let’s call it the mother function. § We apply three operations to it: § Lateral shift φ(t+a) to focus fitting power near a position a. § Scale change φ(bt), to increase resolving power, or the capacity for local curvature. § Smoothing, to increase differentiability and smoothness.

Can I see an example? Consider the Fourier series: § § Mother functions: φ(t) = sin(ωt), and φ(t) = 1 Lateral Shift: sin[ω(t+π/2)] = cos(ωt) Scale change: sin(kωt), cos(kωt), k=1, 2, … Smoothing: None; these are already infinitely differentiable functions. § Strengths: Orthogonality makes computing coefficients very fast § Limitations: Fourier series is only natural if f(t) is periodic with period ω.

Five Fourier Basis Functions § φ1(t) = 1 § A sine/cosine pair with period 1: Captures low frequency variation § A sine/cosine pair with period ½: Captures higher frequency variation

What about polynomials? § The mother function is φ(t) = 1. § Smoothing: Multiply by t to increase both § § § differentiability and curvature. No shift or scaling: Polynomials are invariant with respect to shift and scaling. Strengths: Great for very simple curves such as linear or slight curvature. Limitations: Just not flexible enough. We can’t focus fitting power on a specific location.

What basis should I use for nonperiodic functions? Spline bases have nearly everything: § Infinite and independent control over shift, scale, and smoothness. § Potential to model sharp changes in f(t) as well as its smooth variation. § As fast as polynomials to evaluate. § Band-structured matrix of values.

How do B-splines work? § Start with a discrete mesh of values, called knots: ξ 0, ξ 1, …, ξL. § Space out the knots where you don’t need much resolution, make them dense where you need a lot of resolution.

§ Mother function is the box function over [ξ 0, ξ 1]: φ(t) = 1, ξ 0 ≤ t < ξ 1, and 0 otherwise. § Shift: just shift to another interval. § Scale: determined by spacing between knots.

What about smoothness? § Smoothness control is the heart of spline technology. § It is defined by a recursion relation. § The recursion relation uses spline basis functions at a given smoothness level to make new ones at the next higher level.

B-spline notation The spline basis system we use is the B-spline system. § To keep notation simple, let the knots be the natural numbers j = 0, 1, … § We need two indices for a B-spline basis function Bjm(t): j to indicate the knot where it starts, and m to indicate its order or degree of smoothness. § For our box functions, m = 1.

§ B-splines of order 2 are tent functions, starting at a knot, rising linearly to 1 at the next knot, and decaying linearly to 0 two knots over. § They are continuous. Order 2 means a continuous derivative of order 0. § Order 2 knots are piecewise linear. § What about order 3 Bsplines?

The recursion for integer knots For the B-spline function of order 2 beginning at 0, the recursion is

• In the first term, the mother box function B 01(t) is multiplied by t, and is therefore t itself. • In the second, the mother box is shifted to the right and multiplied by 2 -t. • The second term amounts to a shift and reflection of the first term.

Constructing a Tent B 02(t) from Two Boxes B 01(t) and B 11(t)

Constructing Order 3 B 03(t) from Two Tent Functions

§ Multiplying by (t-j) increases the polynomial degree of the B-spline in the first term by one. § Multiplying (m+j-t) does the same thing for the reflected and shifted B-spline in the second term. § The result is a new B-spline of one degree or order higher, spread over an additional interval, and having an one more level of continuous derivatives. § Order 3 B-splines are piece-wise quadratic, spread over 3 intervals, and have a continuous first derivative.

How do we get the extra smoothness? § t. B 01(t) and (2 -t)B 11(t) have the same values at knot 1. The sum is continuous. § t. B 02(t) and (3 -t)B 12(t) have the same derivative values at knots 1 and 2. The sum has a continuous first derivative. § Each time we multiply by t, and then shift and reflect the result, we add another order of smooth derivative.

Suppose I don’t want so much smoothness at a fixed point? § You can have multiple knots at a point. § For each additional knot, the spline function will have one less derivative at that knot. § An order m Bspline with m knots at a point can be discontinuous at that point. § Most spline software places m knots at each of the boundaries.

How many B-spline basis functions am I using? § The total number of B-spline basis functions is equal to the number of interior knots plus the order. § When there are no interior knots, splines become polynomials: The order equals the number of basis functions.

Wavelets § Wavelets take this shift/scale/smooth trio of operations even further. § They define basis functions organized into orthogonal levels of resolution. § The resulting multi-resolution analysis is one of the most exciting developments in mathematics since Fourier’s series.

Some advice § If you have periodic data, you’ll probably use Fourier series. § If you don’t, you’ll probably use B-splines. § Especially if you need to look at or work with derivatives. § If you want a derivative of order m, fit the data with a spline of order at least m+2, although order 2 m has some advantages.

Where have we been? § Linear combinations of basis functions are a § § convenient way to model functions. The more basis functions we use, the more complex the fitted function can be. Most basis function systems involve turning three knobs, marked shift, scale, and smooth, to modify a mother function. The best systems permit independent control of these features. Like splines and wavelets.

Where do we go from here? § Although we can control the resolution of a spline fit by the number of spline basis functions and the spacing of knots, we often need easier more continuous control over smoothness. § Roughness penalties can give us the additional control that we need.