Instructor Shengyu Zhang 1 LP n n n

Instructor: Shengyu Zhang 1

LP n n n Motivating examples Introduction to algorithms Simplex algorithm q q n n On a particular example General algorithm Duality An application to game theory 2

Example 1: profit maximization n 3

How to solve? n 4

Illustrative figures 5

Example 2 n We are managing a network with bandwidth as shown by numbers on edges. q n 3 connections: AB, BC, CA q q n Bandwidth: max units of flows We get $3, $2, $4 for providing them respectively. Two routes for each connection: short and long. Question: How to route the connections to maximize our revenue? 6

Example 2 n 7

LP in general n Superscript T: transpose of vectors. Inequality: entry-wise 8

Transformations between forms n 9

Transformations between forms n 10

feasibility n 11

Adding the objective function into the picture n The objective function is also linear q n also a line for a fixed value. Thus the optimization is: try to move the line towards the desirable direction s. t. the line still intersects with the feasible region. 12

Possibilities of solution n 13

Three Algorithms for LP n Weakly polynomial time 14

Simplex method: geometric view n 15

Simplex algorithm: Framework n A sequence of (simplex) tableaus n What’s a tableau? 1. Pick an initial tableau 1. How? 2. What’s the rule? 3. When to terminate? Why optimal? 2. 3. Update the tableau Terminate Complexity? 16

An introductory example n n 17

An introductory example n n 18

An introductory example n n 19

An introductory example n n 20

An introductory example n n 21

An introductory example n n 22

An introductory example n n 23

An introductory example n n 24

An introductory example n n 25

An introductory example n n 26

Formal treatment n Now we make the intuitions formal. n We will rigorously define things like basis, feasible basis, tableau, … discuss the pivot steps, and formalize the above procedure for general LP. n n 27

Basis n 28

(Simplex) tableau n 29

n 30

n 31

n 32

Updating n 33

n 34

Picking the initial feasible solution n 35

Solve the new LP first n 36

Simplex Alg: putting everything together n 37

Simplex Algorithm: continued n 38

Efficiency n 39

Theory of simplex method n 40

Duality n n 41

Duality n n This is another linear programming problem. --- dual of the original LP. 42

Making it formal n n 43

44

45

Strong duality n n n The primal gives lower bounds for the dual The dual gives upper bounds for the primal [Strong duality] For linear programming, optimal primal value = optimal dual value q q If both exist, then they are equal If one is infinity, then the other is infeasible 46

A physical interpretation of duality n 47

A physical interpretation of duality n 48

A physical interpretation of duality n 49

A physical interpretation of duality n 50

A physical interpretation of duality n 51

Application: Zero-sum game n 0 1 -1 -1 0 1 1 -1 0 52

Who moves first? n 53

Mixed strategy n 54

Minimax n 55

Proof by LP duality n n Observation: These two LP’s are dual to each other. Thus they have the same optimal value. 56

Summary n n Linear program: a very useful framework Algorithms: q q q n Simplex: exponential in worst-case, efficient in practice. Ellipsoid: polynomial in worst-case but usually not efficient enough for practical data. Interior point: polynomial in worst-case and efficient in practice. Duality: Each LP has a dual LP, which has the same optimal value as the primal LP if both are feasible. 57

References n Our introduction to LP largely follows the book Understanding and Using Linear Programming, Jiři Matoušek and Bernd Gärtner, Springer, 2006. n Many references on LP or other optimization theories. Convex Optimization, Boyd and Vandenberghe, Cambridge University Press, 2004. 58