Computational Physics Lecture 9 PHY 4061 Quadratic form

Computational Physics (Lecture 9) PHY 4061

Quadratic form •

Contour map

The gradient •

Revisit of Steepest descent • We start from an arbitrary starting point: x 0, • Slide down to the bottom of the paraboloid. • When we take the step, we choose the direction in which f decreases most quickly. – -f’(xi)=b-Axi. • Error : ei = xi-x => how far from the solution. • Residual: ri = b-Axi => how far from the correct value of b.

• Suppose we start from (-2, -2). Along the steepest descent, will fall somewhere on the solid line. – X 1 = x 0+αr 0 – How big the step we should take? – Line search • Choose α to minimize f. df(x 1)/dα=0, • So, f’(x 1)Tr 0=0 • So α is chosen to make r 0 and f’(x 1) orthogonal!

Determine α • • F’(x 1)= -r 1 r 1 Tr 0 = 0 (b-Ax 1)Tr 0 = 0 (b-A(x 0+ αr 0))Tr 0 = 0 (b-Ax 0)Tr 0 = α(Ar 0)Tr 0= αr 0 T(Ar 0) α =(r 0 Tr 0)/ (r 0 TAr 0)

General method of Steepest Descent

Any problems can you perceive?

Conjugate directions • SD often takes steps in the same direction! • If we take n orthogonal steps, d 0, d 1 … dn-1, each step has the correct length, after n steps, we are done！ • In general, for each step, we have – xi+1=xi+αidi – ei+1 should be orthogonal to di – di. Tei+1 = 0 (ei+1 = xi+1 -x= ei+ αidi) – di. T (ei+ αidi)=0 – αi=-di Tei/(d i Td i)

• Useless! We don’t know ei. • Instead, we make di and dj A-orthogonal, or conjugate. – di TAdj = 0

• New requirement: – e(i+1) is A-orthogonal to d(i) – Like SD, we need to find the minimum along d(i) Here, xi+1=xi+αidi A e i+1 = A (xi+1 –x) = Axi+1 -Ax=Axi+1 -b = -ri+1

• This time, evaluate the α(i) again, A e i = A (xi –x) = Axi-Ax=Axi-b = -ri

Conjugate Gradients • Reading material: – https: //www. cs. cmu. edu/~quake-papers/painlessconjugate-gradient. pdf • Just the Conjugate directions method where the search directions are constructed by conjugation of the residuals. – ui=r(i)

General Method

Sample code for SD

Revised Algorithm using line minimization for nonlinear functions