Nonlinear systems Newtons method The steepest descent method

Nonlinear systems Newton’s method The steepest descent method 數值方法 2008, Applied Mathematics NDHU 1

Nonlinear systems A general system of n nonlinear equations in n unknowns 數值方法 2008, Applied Mathematics NDHU 2

Example 數值方法 2008, Applied Mathematics NDHU 3

myfun function F = myfun(x) F(1) = 3*x(1)-cos(x(2)*x(3))-1/2; F(2) = x(1). ^2 -81*(x(2)+0. 1). ^2+sin(x(3))+1. 06; F(3) = exp(-x(1)*x(2))+20*x(3)+1/3*(10*pi-3); return 數值方法 2008, Applied Mathematics NDHU 4

數值方法 2008, Applied Mathematics NDHU 5

Solve nonlinear systems x 0= rand(1, 3)-0. 5; x = lsqnonlin(@myfun, x 0); y=myfun(x); sum(y. ^2); 數值方法 2008, Applied Mathematics NDHU 6

Demo_ex 1 demo_ex 1. m 數值方法 2008, Applied Mathematics NDHU 7

Newton’s method -Tangent line y=f(x) yn=f(xn) xn+1 數值方法 2008, Applied Mathematics NDHU xn x 8

Updating rule 數值方法 2008, Applied Mathematics NDHU 9

Newton’s method 數值方法 2008, Applied Mathematics NDHU 10

Flow Chart n=0; n=0 F guess x 0 Halting condition T 數值方法 2008, Applied Mathematics NDHU 11

Matlab coding Matlab modules: 1. 3 -variable function evaluation 2. Use symbolic differentiation to determine partial derivatives 3. Evaluate partial derivatives 4. Main program: use a while loop to update x iteratively 數值方法 2008, Applied Mathematics NDHU 12

Function evaluation s 1='3*x 1 -cos(x 2*x 3)-1/2'; f=inline(s 1); x=rand(1, 3); y=f(x(1), x(2), x(3)); 數值方法 2008, Applied Mathematics NDHU 13

Evaluation of vector function s 1='3*x 1 -cos(x 2*x 3)-1/2'; s 2='x 1. ^2 -81*(x 2+0. 1). ^2+sin(x 3)+1. 06'; s 3='exp(-x 1*x 2)+20*x 3+1/3*(10*pi-3)'; f 1=inline(s 1); f 2=inline(s 2); f 3=inline(s 3); x=rand(1, 3); Y=f 1(x(1), x(2), x(3)); Y=[Y f 2(x(1), x(2), x(3))]; Y=[Y f 3(x(1), x(2), x(3))]; 數值方法 2008, Applied Mathematics NDHU 14

Variant function form s='x 1+x 2'; f=inline(s); x=rand(1, 3); y=f(x(1), x(2), x(3)); s='x 1+x 2'; f=inline(s); x=rand(1, 3); y=f(x(1), x(2)); Error message: Too many input to inline function Correct codes 數值方法 2008, Applied Mathematics NDHU 15

s='x 1+x 3'; f=inline(s); x=rand(1, 3); y=f(x(1), x(3)); s='x 2+x 3'; f=inline(s); x=rand(1, 3); y=f(x(2), x(3)); Correct codes 數值方法 2008, Applied Mathematics NDHU 16

s='1'; f=inline(s); x=rand(1, 3); y=f(0); Correct codes s='x 1+x 2+x 3'; f=inline(s); x=rand(1, 3); y=f(x(1), x(2), x(3)); Correct codes 數值方法 2008, Applied Mathematics NDHU 17

symvar sc=symvar(f); l f denotes an inline function l sc is a cell structure l Symbols in f 數值方法 2008, Applied Mathematics NDHU 18

Symbols in an inline function s='x 1+x 2'; f=inline(s); s='x 1+x 3'; f=inline(s); s='1'; f=inline(s); s='x 2+x 3'; f=inline(s); s='x 1+x 2+x 3'; f=inline(s); sc=symvar(f); sc = 'x 1' 'x 2' y=f(x(1), x(2)); sc = 'x 2' 'x 3' 'x 1' 'x 3' y=f(x(1), x(3)); y=f(x(2), x(3)); 'x' y=f(0); 數值方法 2008, Applied Mathematics NDHU sc = 'x 1' 'x 2' 'x 3' y=f(x(1), x(2), x(3)); 19

3 -variable function evaluation xx=fun 3 v(f) f: an inline function l xx: a string for 3 -variable function evaluation l 數值方法 2008, Applied Mathematics NDHU 20

Newtonfun 3 v. m function xx=fun 3 v(f) s=symvar(f); xx=[]; for i=1: size(s, 1) if i>1 xx=[xx ', ']; end arg=char(s(i)); switch arg case 'x' xx=[xx '0']; case 'x 1' xx=[xx 'x(1)']; case 'x 2' xx=[xx 'x(2)']; case 'x 3' xx=[xx 'x(3)']; end xx=['f(' xx ')']; return 數值方法 2008, Applied Mathematics NDHU 21

example s='x 1+x 2'; f=inline(s); s='x 1+x 3'; f=inline(s); s='x 2+x 3'; f=inline(s); s='1'; f=inline(s); s='x 1+x 2+x 3'; f=inline(s); xx=fun 3 v(f) x=[1 2 3] eval(xx) 數值方法 2008, Applied Mathematics NDHU 22

3 -variable function evaluation function v=feval 3 v(f, x) xx=fun 3 v(f); eval(xx) return 數值方法 2008, Applied Mathematics NDHU 23

Matlab codes feva. m function v=feva(f, x) s=symvar(f); xx=[]; for i=1: size(s, 1) ss=char(s(i)); switch ss case 'x' xx=[xx 0]; case 'x 1' xx=[xx x(1)]; case 'x 2' xx=[xx x(2)]; case 'x 3' xx=[xx x(3)]; end xx=['f(' xx ')']; v=eval([sprintf(xx)]); return 數值方法 2008, Applied Mathematics NDHU 24

Partial derivation s='3*x 1 -cos(x 2*x 3)-1/2'; fx=inline(s); x 1=sym('x 1'); x 2=sym('x 2'); x 3=sym('x 3'); J 1=inline(diff(s, x 1)) J 2=inline(diff(s, x 2)) J 3=inline(diff(s, x 3)) 數值方法 2008, Applied Mathematics NDHU 25

function [J 1, J 2, J 3]=pdiff(s) Newtonpdiff. m 數值方法 2008, Applied Mathematics NDHU 26

Example s='3*x 1 -cos(x 2*x 3)-1/2'; [J 1 J 2 J 3]=pdiff(s) J 1 = Inline function: J 1(x) = 3 J 2 = Inline function: J 2(x 2, x 3) = sin(x 2. *x 3). *x 3 J 3 = Inline function: J 3(x 2, x 3) = sin(x 2. *x 3). *x 2 數值方法 2008, Applied Mathematics NDHU 27

Partial derivatives s 1='3*x 1 -cos(x 2*x 3)-1/2'; s 2='x 1^2 -81*(x 2+0. 1)^2+sin(x 3)+1. 06'; s 3='exp(-x 1*x 2)+20*x 3+1/3*(10*pi-3)'; [J 1 J 2 J 3]=pdiff(s 1); [J 4 J 5 J 6]=pdiff(s 2); [J 7 J 8 J 9]=pdiff(s 3); 數值方法 2008, Applied Mathematics NDHU 28

Newtondemo. m 數值方法 2008, Applied Mathematics NDHU 29

Function x=Newton(s 1, s 2, s 3, x 0) s 1: string of function f 1 s 2: string of function f 2 s 3: string of function f 3 x 0: initial guess 數值方法 2008, Applied Mathematics NDHU 30

% % Determine F(1), F(2), F(3) by substituting x to f 1, f 2 and f 3 % % Determine J by substituting x to J 1, J 2, …, J 9 % % Update x 數值方法 2008, Applied Mathematics NDHU 31

Exercise Write matlab codes to implement the Newton’s method for solving a threevariable nonlinear system l Test your matlab function with the following nonlinear system l 數值方法 2008, Applied Mathematics NDHU 32

The steepest descent method is translated to minimize 數值方法 2008, Applied Mathematics NDHU 33

Gradient 數值方法 2008, Applied Mathematics NDHU 34

Iterative process 數值方法 2008, Applied Mathematics NDHU 35

數值方法 2008, Applied Mathematics NDHU 36

Newton’s method Steepest gradient descent method 數值方法 2008, Applied Mathematics NDHU 37

Steepest descent l Choose to minimize 數值方法 2008, Applied Mathematics NDHU 38

數值方法 2008, Applied Mathematics NDHU 39

Quadratic polynomial: 數值方法 2008, Applied Mathematics NDHU 40

數值方法 2008, Applied Mathematics NDHU 41

Function x=Steepest(s 1, s 2, s 3, x 0) Create three nonlinear inline functions Create nine inline functions for partial derivatives of the three nonlinear functions Set x to x 0 While the halting condition is not satisfied 1. 2. 3. 4. 5. Substitute x to all partial derivatives to form a 3 x 3 matrix J Substitute x to the three nonlinear functions Find optimal Update x by the steepest descent method Return x 數值方法 2008, Applied Mathematics NDHU 42