Chapter 3 Root Finding 1 3 1 The

Chapter 3 Root Finding 1

3. 1 The Bisection Method ► Let f be a continues function. Suppose we know that f(a) f(b) < 0, then there is a root between a and b. 2

Example 3. 1 ► A formal statement is given in Algorithm 3. 1. 3

4

Theorem 3. 1 Bisection Convergence and Error 5

6

Bisection Method ► Advantage: § A global method: it always converge no matter how far you start from the actual root. ► Disadvantage: § It cannot be used to find roots when the function is tangent is the axis and does not pass through the axis. ► For example: § It converges slowly compared with other methods. 7

3. 2 Newton’s Method: Derivation and Examples ► Newton’s method is the classic algorithm for finding roots of functions. ► Two good derivations of Newton’s method: § Geometric derivation § Analytic derivation 8

Newton’s Method : Geometric Derivation 9

Newton’s Method : Geometric Derivation The fundamental idea in Newton’s method is to use the tangent line approximation to the function f at point. ► The point-slope formula for the equation of the straight line gives us: ► ► Continue the process with another straight line to get 10

Newton’s Method : Analytic Derivation 11

Example 3. 2 12

13

Newton’s Method ► Advantage: § Very fast ► Disadvantage: § Not a global method ► For example: Figure 3. 3 (root x = 0. 5) ► Another example: Figure 3. 4 (root x = 0. 05) ► In these example, the initial point should be carefully chosen. § Newton’s method will cycle indefinitely. ► Newton’s method will just hop back and forth between two values. ► For example: Consider (root x = 0) 14

15

Initial value Wrong predictions, because the root is positive Very close to the actual root 16

3. 3 How to Stop Newton’s Method ► Ideally, we would want to stop when the error sufficiently small. is (p. 12) 17

To make sure f(xn) is also small enough 18

3. 4 Application: Division using Newton’s Method ► The purpose is to illustrate the use of Newtown’s method and the analysis of the resulting iteration. f (x) f ’(x) 19

► Questions: § When does this iteration converge and how fast? § What initial guesses x 0 will work for us? ► The way that computer stores numbers: 20

21

From (2. 11) p. 53 Initial x 0 p. 56 22

Example 3. 3 23

3. 5 The Newton Error Formula 24

Definition 3. 1 The requirement that C be nonzero and finite actually forces p to be a single unique value. ► Linear convergence: p = 1 ► Quadratic convergence: p = 2 ► Superlinearly convergence: but ► 25

Example 3. 6 26

3. 6 Newton’s Method: Theory and Convergence ► Its proof is shown at pp. 106 -108. 27

3. 7 Application: Computation of the Square Root 28

► Questions: § Can we find an initial guess such that Newton’s method will always converge for b on this interval? § How rapidly will it converge? ► The Newton error formula (3. 12) applied to : (3. 25) ► The relative error satisfies (3. 26) 29

relative error 30

How to find the initial value? ► Choose the midpoint of the interval § For example: If ► Using , linear interpolation § For example: b is known 31

3. 8 The Secant Method: Derivation and Examples ► An obvious drawback of Newton’s method is that it requires a formula for the derivative of f. ► One obvious way to deal with this problem is to use an approximation to the derivative in the Newton formula. § For example: ► Another method: the secant method § Used a secant line 32

The Secant Method 33

The Secant Method 34

The Secant Method ► Its advantages over Newton’s method: § It not require the derivative. § It can be coded in a way requiring only a single function evaluation per iteration. ► Newton’s requires two, one for the function and one for the derivative. 35

Example 3. 7 36

37

Error Estimation ► The error formula for the secant method: 38

The Convergence ► This is almost the same as Newton’s method. 39

3. 9 Fixed-point Iteration The goal of this section is to use the added understanding of simple iteration to enhance our understanding of and ability to solve rootfinding problems. ► The root of f is equal to the fixed-point of g. ► root 40

Fixed-point Iteration ► Because show that this kind of point is called a fixed point of the function g, and an iteration of the form (3. 33) is called a fixed-point iteration for g. 41

Fixed point Root 42

43

44

45

Example 3. 8 46

47

g (x) 48

Theorem 3. 5 49

Theorem 3. 5 (con. ) 50

51

3. 10 Special Topics in Root-finding Method ► 3. 10. 1 Extrapolation and Acceleration § The examples have some mistakes, so we jump this subsection. 52

3. 10. 2 Variants of Newton’s Method ► Newton’s method v. s. the chord method v. s. ► How much do we lose? § The chord method is only linear, and only locally convergent. ► The chord method is useful in solving nonlinear systems of equations. ► One interesting variant of the chord method updates the point at which the derivative is evaluated, but not every iteration. 53

Example 3. 12 54

55

Other Approximations to the Derivative ► In Section 3. 8, a method using a finite difference approximation to the derivative in Newton’s method. Only linear convergence (shown on pages 133 to 134) 56

3. 10. 3 The Secant Method: Theory and Convergence The proof is shown on pages 136 to 139. You can study it by yourselves. 57

3. 10. 4 Multiple Roots ► So far our study of root-finding methods has assumed that the derivative of the function does not vanish at the root: ► What happens if the derivative does vanish at the root? 58

Example 3. 13 -1 59

L’Hopital’s Rule forms of type 0/0 60

Another example (f(x)=1 -xe 1 -x) 61

Another example (f(x)=1 -xe 1 -x) ► The data (Table 3. 10 a) suggests that both iterations are converging, but neither one is converging as rapidly as we might have expected. ► Can we explain this? § The fact that will have an effect on both Newton and secant methods. § The error formulas (3. 12) and (3. 50) and limits (3. 24) and (3. 47) all require that. § Can we find anything more in the way of an explanation? 62

Discussion—Newton’s Method ► Assume f has a double root and ► Note that we no longer have , therefore (according to Theorem 3. 7 page 124) we no longer have quadratic convergence. 63

Discussion—Newton’s Method ► If we change the Newton iteration to be now we have . (quadratic convergence) ► More generally, ► The problem with this technique is that it requires that we know the degree of multiplicity of the root ahead of time. 64

Discussion—Newton’s Method ► So an alternative is needed. ► The draw back of this method is that applying Newton’s method to u will require that we have a formula for the second derivative of f. 65

Discussion—Newton’s Method ► (3. 60) ► (3. 61) 66

Table 3. 10 67

Discussion ► From Table 3. 10, we can see the accuracy is not as good as past. What is going on? ► Let’s look at a graph of the polynomial Fig. 3. 11 shows a plot of 8000 points from this curve on the interval [0. 45, 0. 55] (root = 0. 5) Premature convergence ► This is not caused by the root-finding method. It is because using finite precision arithmetic. 68

69

3. 10. 5 In Search of Fast Global Convergence: Hybrid Algorithm ► Bisection method: § slow but steady and reliable ► Newton’s method and the secant method: § fast but potentially unreliable ► Brent’s algorithm: § incorporate these basic ideas into an algorithm § Algorithm 3. 6 70

71

Example 3. 14 Step 1. Step 3. (b) Step 3. (c) Step 2. (a) Step 1. Step 2. (b) 72

Another Example 73