Column Generation By Soumitra Pal Under the guidance

Column Generation By Soumitra Pal Under the guidance of Prof. A. G. Ranade

Agenda • • Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

Cutting Stock Problem 10 • Given larger raw paper rolls • Get final rolls of smaller widths 5 3

Cutting Stock Problem (2) • Raw width (W) = 10 • No of finals given below i Width (wi) Quantity (bi) 1 3 9 2 5 79 3 6 90 4 9 27 • Minimize total no of raws reqd. to be cut

Agenda • • Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

Objective/ Cost fn Constraints Min -x 1 -x 2 5 x 1 -5 x 1 + 8 x 2 4 x 2 2 x 2 ≥ -80 ≥ -100 ≥ -80 ≥ -50

5 x 1 -5 x 1 + (8, 15) (0, 10) (12, 10) (13, 5) (10, 0) 8 x 2 4 x 2 2 x 2 ≥ -80 ≥ -100 ≥ -80 ≥ -50

Cost decreases in this direction -x 1 -x 2 = -10 -x 1 -x 2 = -20 -x 1 -x 2 = -30

Simplex method (8, 15) (0, 10) (12, 10) (13, 5) (10, 0)

Basic & non-basic 5 x 1 -5 x 1 + 8 x 2 4 x 2 2 x 2 ≥ -80 ≥ -100 ≥ -80 ≥ -50 (8, 15, 0, 0, 10, 40) (0, 10, 0, 60, 70) (12, 10, 60, 0, 0, 10) (13, 5, 110, 0, 0) (0, 0, 80, 100, 80, 50) (10, 0, 130, 50, 30, 0)

• If all elements of it are non-negative, it is already optimal • Otherwise choose the non-basic variable corresponding to most negative value to enter the basic

• How to find outgoing basic variable?

Few steps of simplex algorithm • Compute reduced cost • If all >= 0, optimal stop. • Otherwise choose the most negative component • Corresponding variable enters basis • Compute the ratios for finding out leaving basis • Update basic variables and B for next step

Smart way of doing

Start d y

Reduced cost vector d y

Selection of non-basic variable d y

y vector d y

Selection of leaving basic variable d y

Preparation for next step d y

Entering variable in 2 nd iteration d y

Leaving variable in 2 nd iteration d y

Preparation for 3 rd iteration d y

Final iteration d y

Done so far • Basics of (revised) Simplex algorithm – Formulation – Corners, basic variables, non-basic variables – How to move from corner to corners – Optimal value

Agenda • • Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

First step to solution - Formulation Kantorovich formulation

Kantorovich formulation example 1 2 3 4

LP relaxation • Integer programming is hard • Convert it to a linear program 0 ≤ yk ≤ 1

LP relaxation (2) • • • LP relaxation is poor For our example, optimal LP objective is 120. 5 The optimal integer objective solution value is 157 Gap is 36. 5 It can be as bad as ½ of the integer solution • Can we get a better LP relaxation?

Another Formulation Gilmore-Gomory Formulation

Example formulation 3*1 + 5*0 + 6*1 + 9*0 = 9 ≤ 10

LP relaxation of Gilmore-Gomory • • • LP relaxation is better For our example, 156. 7 Integer objective solution, 157 Gap is 0. 3 Conjecture: Gap is less than 2 for practical cutting stock problems

Issues • The number of columns are exponential • Optimal value is still not integer • Gilmore-Gomory proposed an ingenuous way of working with less columns • Branch-and-price to solve the other problem

Done so far • Basics of (revised) Simplex algorithm – Formulation – Corners, basic variables, non-basic variables – How to move from corner to corners – Optimal value • Formulation of the CSP – LP relaxation – Bounds

Agenda • • Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

Column Generation • In pricing step of simplex algorithm one basic variable leaves and one non-basic variable enters the basis • This decision is made from the reduced cost vector • The component with most negative value determines the entering variable

Column Generation (2) • In terms of columns, we need to find • That is equivalent to finding a such that • For cutting stock problem, cj=1, hence it is equivalent to • With implicit constraint • Pricing sub-problem

Column Generation (3) Solve Restricted Master Problem (RMP) Update RMP with New Columns Solve Pricing Problem Yes Any New Columns? No STOP (LP Optimal)

Example formulation

Done so far • Basics of (revised) Simplex algorithm – – Formulation Corners, basic variables, non-basic variables How to move from corner to corners Optimal value • Formulation of the CSP – LP relaxation – Bounds • Delayed column generation – Restricted master problem – Sub-problem to generate column – Repeated till optimal

Agenda • • Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

Integer solution • A formulation of the problem which gives ‘good’ LP relaxation • Solved the LP relaxation using column generation • But, the LP relaxation is not the ultimate goal, we need integer solution • In the example x 2 is 39. 5 • One way is to rounding

• What to do if there are multiple fractional variables? • The method commonly followed is branch -and-bound technique of solving integer programs • Basic fundae – create multiple sub-problems with extra constraints – solve the sub-problems using column generation – take the best of solutions to the sub problems

• Key is the rules of dividing into sub problems (this is called branching strategy) • Things to look into – Branching rule does not destroy column generation sub problem property – Efficiency of the process • Ingenuity of the formulation and branching strategy • Use of running bounds to prune (fathomed) • This technique is called branch-and-price • We see another formulation next

Done so far • Basics of (revised) Simplex algorithm – – Formulation Corners, basic variables, non-basic variables How to move from corner to corners Optimal value • Formulation of the CSP – LP relaxation – Bounds • Delayed column generation – Restricted master problem – Sub-problem to generate column – Repeated till optimal • Branch-and-price – Basic fundae

Agenda • • Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

Flow formulation • Bin packing problem • Similar to the cutting stock problem bin items

Example • Bin capacity W = 5 • Item sizes – w 1=3 and w 2=2 w 2 loss 0 w 2 loss 1 w 2 loss 4 3 w 1 w 2 5 w 1 w 2 loss 0 1 2 3 4 5

Formulation equations • Problem is equivalent to finding minimum flow between nodes 0 & W

• • Issue of symmetry in the graph 3 rules to reduce symmetry 1. Arcs are specified according to decreasing width 2. A bin can never start with a loss 3. Number of consecutive arcs corresponding to a single item size must not exceed number of orders

Invalid as per rule 1 Invalid as per rule 2 w 2 loss 0 w 2 loss 1 loss 2 loss 4 3 w 1 w 2 loss 0 w 2 1 2 w 1 w 2 loss 3 loss 4 5 5

Branch-and-price • The sub-problem finds an longest unit flow path where cost of each arc is given by y vector • Branching heuristics – Xij >= ceil(xij) – Xij <= floor(xij)

Few other points • Loss constraints from the LP relaxation value is equated with the sum of loss variables • With the above constraint it can be shown that the solution will be always integral • The algorithm uses this fact by incrementally adding the lower bound • This is done finite no. of times • No of new constraints are added is finite • Algorithm runs in pseudo-polynomial time

Conclusions & Future work • Column generation is a success story in large scale integer programming • The flexibility to work with a few columns make real life problems tractable • Ingenuous formulations and branching heuristics are used to improve efficiency • Need to explore more applications with different heuristics

Acknowledgements • Valerio de Carvalho for borrowing some of his explanations exactly • http: //wwwfp. mcs. anl. gov/otc/Guide/Case. Studies/s implex/applet/Simplex. Tool. html for the java applet • AMPL & CPLEX tools • My guide

Thank you!