Honors Track Competitive Programming Problem Solving Programming Contests

Honors Track: Competitive Programming & Problem Solving Programming Contests Kevin Verbeek

Preparation Study n Standard algorithms n Algorithmic techniques n Common errors Train n Practice many problems n Try diverse problems Prepare n The cheatsheet! (≤ 25 pages) n Sleep the night before… Keep in mind n It is supposed to be fun!

Tactics The contest has started. What now? p One of you should make a template n Without bugs! p Quickly read through all problems n You can read the problems carefully later n Try to assess the difficulty of a problem n Start with the first encountered problem that is easy n Solve problems from easy to hard n Assessing difficulty is hard… Hint: Check the scoreboard. The easiest problems are often solved first. Hint: Easier problems are often shorter… but not always.

Teamplay p 1 computer, 3 contestants ➨ Teamplay is important! p Tactics depend on skills of team members n Is everyone a decent programmer? n Does anyone have any specialties? p One might be good with DP, another with graph algorithms n Is someone good at annoying other teams? p Think before you code! (solve the problem completely) n Unless nobody is using the computer p The rest (those who are not coding) n Works on another problem, or n Comes up with hard test-cases/boundary cases for the problem Hint: If computer is occupied when you need it, start pencoding!

Algorithms Standard algorithms 1. Basics 2. Mathematics 3. Greedy 4. Dynamic Programming 5. Brute force/backtracking 6. String matching 7. Simulations/bookkeeping/compiling 8. Graph algorithms 9. Computational geometry/topology

Basics Basic algorithms n Sorting (use standard library) n Binary search Binary. Search(A, v) 1. x = 0 2. y = n + 1 3. while x + 1 < y 4. do h = floor((x + y)/2) 5. do if A[h] ≤ v then x = h 6. else y = h 7. if A[x] = v then return x else return NIL Hint: Always check if sorting the input simplifies the problem Hint: For optimization problems, check if a binary search helps

Mathematics Algebra n You sometimes need some basic algebra/calculus n Cannot teach you all of this… Prime numbers n Primality test is easy (check until √n of course) n Sometimes you need to generate all prime numbers ≤ n p Use Sieve of Eratosthenes! O(n log n) Other basics n GCD/LCM, useful for implementing rational numbers n Modular arithmetic in a field Z/p. Z n Generating permutations (for brute-force algorithms) n … or combinations, etc.

Mathematics Linear algebra n You need to know Gaussian elimination n Recognizable by linear equations in n unknowns n Beware of cases without a unique solution Transformations n Translations / Rotations / Scaling n Pretty basic in 2 D… n Also doable in 3 D Hint: Use complex numbers for 2 D scaling and rotation

Algorithms Standard algorithms 1. Basics 2. Mathematics 3. Greedy 4. Dynamic Programming 5. Brute force/backtracking 6. String matching 7. Simulations/bookkeeping/compiling 8. Graph algorithms 9. Computational geometry/topology

Greedy algorithms How to recognize? n Tricky n Often easy optimization problems Difficulty n Very easy, both the algorithm and the implementation n But you must prove correctness! n Why is the greedy choice optimal? Hint: Usually a greedy approach doesn’t work!

Dynamic programming Standard dynamic programming algorithms n (0/1)-knapsack n Edit distance n Longest common subsequence n Coin change n Longest increasing subsequence n Matrix chain multiplication n Minimum cost convex polygon triangulation 1 7 3 2 3 9 4 7 11

Dynamic programming How to recognize? n Optimization problems (or counting problems) n As input you often get arrays/tables of values n They are often similar to standard DP problems Difficulty n Coming up with algorithm (recursive relation) can be hard n Implementation is often very easy n Just a matter of filling in a table What’s important? n Make your arrays big enough! n Be careful with loop bounds and array indices!

Brute force/backtracking How to recognize? n These problems are often NP-hard p But then you need to be familiar with NP-hard problems n Small input bounds are a dead giveaway Difficulty n As soon as it is clear you can use brute force, it’s relatively easy n Only problem is to generate all possibilities n This is often easiest with a recursive function What’s important? n That you really generate all possibilities, but not many more n Often you need backtracking to be fast enough

Algorithms Standard algorithms 1. Basics 2. Mathematics 3. Greedy 4. Dynamic Programming 5. Brute force/backtracking 6. String matching 7. Simulations/bookkeeping/compiling 8. Graph algorithms 9. Computational geometry/topology

String matching How to recognize? n Often very easy Difficulty n If you know the standard algorithms, pretty easy n Quite tricky otherwise… Variants n Given a string S, check if S occurs in a list of strings p Use hashing or a trie n Given a string S, find all occurrences of S in a big string p Use FSM or Boyer-Moore or Knuth-Morris-Pratt What’s important? n Easy to make mistakes. Be careful.

Simulations/bookkeeping/compiling How to recognize? n Often very clear Difficulty n Just do it, not very hard n Often a lot of work, so takes a while. Don’t start with these problems n It’s better to start with shorter problems What’s important? n Lots of code ➨ lots of bugs. Be careful! n Really think before you code n Boundary cases!

Graph algorithms How to recognize? n Often the input is a graph n But not always… Hint: Try to see every problem as a graph problem Difficulty n Can be both easy and hard p DFS/BFS is easy, max-flow is hard n Usually takes quite some time to implement What’s important? n Numbering of vertices n Boundary cases

Graph algorithms Standard Algorithms n DFS n BFS n Single source shortest path (Dijkstra / Bellman-Ford) n All-pairs shortest path (Floyd-Warshall) n MST (Prim/Kruskal) n Euler tour n Bipartite matching n Max-flow (Ford-Fulkerson)

Graph representations n Adjacency matrix ➨ easy, but slow and memory-heavy n Adjacency list ➨ straightforward and efficient C++ Java struct Edge { int target; int weight; … }; class Edge { int target; int weight; … } struct Node { vector<Edge> adj; bool visited; … }; class Node { Array. List<Edge> adj; bool visited; … } Node V[10005]; Node[] V = new Node[10005];

Depth First Search DFS n Easy to implement with recursive function n Running time is O(V + E) DFS(v) { v. visited = true; for v’ in v. adj if (!v’. visited) DFS(v’) } Problems n Connected components (easy) n Strongly connected components (Tarjan’s algorithm) n Biconnected components

Breadth First Search BFS n Slightly harder to implement than DFS n Useful for simple shortest path problems n Easy to implement with a queue n Running time is O(V + E) Grid-BFS n These problems often occur n Almost always solvable using a BFS n You sometimes need to be creative with states Hint: For Grid-BFS, add a wall around the grid ## #### # # # ### # # p# # #######

Shortest Paths Single source shortest path n Also for the shortest path between two points n A common problem during contests Do the edges have different lengths/weights? Yes No (for example for grids) BFS Can the edges have negative weight? No Dijkstra Yes Bellman-Ford

Bellman-Ford n Not very common, but you should know it anyway n Also works with edges with negative weights/lengths n Very easy to implement n Can also be used to find negative cycles n Running time is O(V * E) For i = 1 to V - 1 For each edge e = (u, v) if u. dist + e. weight < v. dist = u. dist + e. weight; v. pred = u;

All-pairs shortest path n If you need the shortest path between every pair of vertices n Also possible with Dijkstra, but not as easy n Running time is O(V 3) Floyd-Warshall path[i][j] = wij; For k = 1 to n For i = 1 to n For j = 1 to n path[i][j] = min(path[i][j], path[i][k] + path[k][j]);

Minimum Spanning Tree Minimum spanning tree n Pretty simple algorithms n Use Prim or Kruskal (Running time O(E log V)) 4 1 Prim 7 1 3 5 8 E’ = {}; V’ = {v}; (arbitrary vertex) 5 While V’ ≠ V Choose minimum-weight edge (u, v) such that u in V’ and v isn’t V’ = V’ U {v}; E’ = E’ U {(u, v)} 6 2 4 Kruskal E’ = {}; Sort E on weight; For every (u, v) in E If E’ U {(u, v)} does not contain a cycle E’ = E’ U {(u, v)} 1 7 1 3 8 5 6 2 5

Computational geometry/topology How to recognize? n You get points, lines, triangles, polygons, etc. Difficulty n Often very hard to implement n Many boundary cases/degeneracies Hint: Comp. geom. problems sometimes look easy, but they’re usually not. Only try to solve these problems if you’re sure you can. What’s important? n Boundary cases! n Floating point precision!

Computational geometry/topology Don’t try these n 3 D convex hull n Voronoi diagram n Nearest-neighbor (unless you know exactly how) n Plane-sweep n Spatial queries You can try these n 2 D convex hull (Graham scan) n Intersections between lines/circles/planes n Area/centroid polygon n Point in polygon n Visibility graph

Computational geometry/topology Computational topology n Euler’s formula can be useful: V – E + F = C + 1 p C = #connected components n Use it when you need to compute the number of faces

Estimating running time Is my algorithm fast enough? n Know the asymptotic running times of standard algorithms n Estimate the worst case number of steps of your program n If #steps < ~10 000, then start implementing Example You’re given a graph with n vertices and m edges with n ≤ 103 and m ≤ 106. Can we use Dijkstra’s algorithm? n Dijkstra’s algorithm runs in O((m+n) log n) time n That is approximately 106 log 103 ≈ 107 steps in worst case n So … Yes!

Estimating algorithm But we can also do it the other way around! n Given the bounds on the input n What could possibly be the asymptotic running time? n Which algorithms run in this time? p n ≤ 109 O(1), O(log n) or O(√n) Function, BS? p n ≤ 106 O(n) or O(n log n) Greedy, sorting, BS + Greedy, D&C? p n ≤ 103, W ≤ 103 O(n. W) dynamic prog. with table n x W? p n ≤ 102 O(n 3) All-pairs shortest path, Gaussian elimination? p n ≤ 16 O(2 n) or O(n 2 n) Brute-force on all bitstrings of length n p n ≤ 8 O(n!) Brute-force on all permutations of n objects

Golden rules of programming 1. Think before you code 2. Always choose the simplest solution that is fast enough 3. Code carefully rather than fast

Implementation After implementation n First test with provided test-cases n If your program doesn’t work on these cases: p Check if the input is read correctly p Use debugger to check intermediary results p If this takes too long, then print & switch n Also test some boundary cases n Then submit! Hopefully you see … Correct! n … but usually you don’t.

Implementation Possible responses n Wrong answer (very annoying) n Runtime error (usually easy to fix) n Time-limit exceeded (depends on why) ➨ print & switch! Time-limit exceeded n Algorithm too slow (shouldn’t happen if you estimated correctly) n Infinite loop: Check if this is possible n Otherwise try simple optimizations Runtime error n Usually array index out of bounds (check that) n Invalid expression, like sqrt(-1) n Out of memory

Common mistakes Wrong Answer 1. Loop bounds n Think carefully about all loop bounds (especially with dynamic prog. ) 2. Array bounds n Check if all arrays are big enough 3. Initialization n Initialize all data again for every testcase (clear vectors/lists/etc. ) n This only gives errors for multiple testcases 4. Incorrect output format n Use correct spelling, capitals and endlines 5. Wrong nesting n Check whether the nesting is correct (use indentation) 6. Precision n Always use doubles when you need floating point numbers n For many multiplications, use log: log (a * b) = log(a) + log(b) n Use the correct precision for output (setprecision)

Common mistakes 7. Overflow n Check how large the numbers can get n Use long or something similar (remember for C: scanf(“%lld”)) 8. Invalid expression n Sqrt(-1) or log(0), etc. 9. Index offset n A[i] or A[i-1]? Often important for dynamic programming 10. Rounding n Read the problem carefully and round as prescribed 11. Read the complete input n Sometimes the answer is clear before you’ve read the complete input n Read it anyway! Output the answer only after reading complete input! 12. Boundary cases n These cases often cause problems, make some testcases 13. Wrong variable / copying mistakes / etc.