The Best Algorithms are Randomized Algorithms N Harvey

The Best Algorithms are Randomized Algorithms N. Harvey C&O Dept

WHY DO WE RANDOMIZE?

Reasons for Randomness • • • Fooling Adversaries Symmetry Breaking Searching a Haystack Sampling huge sets. . .

Picking Passwords • Encryption keys are randomly chosen

The Hallway Dance • Randomly go left or right • After a few twists you won’t collide (usually!)

Quick. Sort 17 5 5 9 23 12 9 4 12 23 4 19 10 17 19 10 • If I knew an approximate median , I could partition into 2 groups & recurse • A random element is probably close to the median • Like “searching for hay in a haystack”

Reasons for Randomness • Fooling Adversaries • Symmetry Breaking • Searching a Haystack • . . .

Another Adversary Example IP • Linux 2. 4. 20 denial-of-service attack • Kernel uses a hash table to cache info about IP traffic flows • Adversary sends many IP packets that all have same hash value • Hash table becomes linked list; Kernel very slow • Solution: Randomized hash function

Another Symmetry Breaking Example • Ethernet Media Access Control • Try to send packet • If collision detected –Wait for a random delay –Retry sending packet http: //picasaweb. google. com/tomatobasil

Another Haystack Example • Consider the polynomial f(x, y)=(x-y 2)(x 2+2 y+1) • Problem: Find a point (x, y) that is not a zero of f • If f 0 then, for randomly chosen (x, y), Pr[ f(x, y)=0 ] = 0 • Even over finite fields, Pr[ f(x, y)=0 ] is small

Reasons for Randomness • Fooling Adversaries • Symmetry Breaking • Searching a Haystack • . . . • Beauty: randomization often gives really cool algorithms !

Graph Basics • A graph is a collection of vertices and edges • Every edge joins two vertices • The degree of a vertex is # edges attached to it

Graph Connectivity A connected graph A disconnected graph • Problem: How many edges do you need to remove to make a graph disconnected?

Graph Connectivity • Problem: How many edges do you need to remove to make a graph disconnected? • Connectivity is minimum # edges whose removal disconnects the graph • Observation: for every vertex v, degree(v) connectivity • Because removing all edges attached to vertex v disconnects the graph

Graph Connectivity • Problem: How many edges do you need to remove to make a graph disconnected? • Connectivity is minimum # edges whose removal disconnects the graph • Useful in many applications: • How many MFCF network cables must be cut to disrupt MC’s internet connectivity? • How many bombs can my oil pipelines withstand?

Graph Connectivity Example S • Removing red edges disconnects graph • Definition: A cut is a set of edges of the form { all edges with exactly one endpoint in S } where S is some set of vertices. • Fact: min set of edges that disconnects graph is always a cut

Algorithms for Graph Connectivity • Given a graph, how to compute connectivity? • Standard Approach: Network Flow Theory – Ford-Fulkerson Algorithm computes an st min cut (minimum # edges to disconnect vertices s & t) – Compute this value for all vertices s and t, then take the minimum – Get min # edges to disconnect any two vertices • CO 355 Approach: – Compute st min cuts by ellipsoid method instead • Next: An amazing randomized approach

Algorithm Overview • Input: A haystack • Output: A needle (maybe) • While haystack not too small – Pick a random handful – Throw it away • End While • Output whatever is left

Edge Contraction wu v • Key operation: contracting an edge uv • Delete the edge uv • Combine u & v into a single vertex w • Any edge with endpoint u or v now ends at w

Edge Contraction u v w • Key operation: contracting an edge uv • Delete the edge uv • Combine u & v into a single vertex w • Any edge with endpoint u or v now ends at w • This can create “parallel” edges

Randomized Algorithm for Connectivity • Input: A graph • Output: Minimum cut (maybe) • While graph has 2 vertices “Not too small” – Pick an edge uv at random “Random Handful” – Contract it “Throw it away” • End While • Output remaining edges

Graph Connectivity Example • We were lucky: Remaining edges are the min cut • How lucky must we be to find the min cut? • Theorem (Karger ‘ 93): The probability that this algorithm finds a min cut is 1/(# vertices)2.

But does the algorithm work? • How lucky must we be to find the min cut? • Theorem (Karger ‘ 93): The probability that this algorithm finds a min cut is 1/n 2. (n = # vertices) • Fairly low success probability. Is this useful? • Yes! Run the algorithm n 2 times. 2 2 n Pr[ fails to find min cut ] (1 -1/n ) 1/e • So algorithm succeeds with probability > 1/2

Proof of Main Theorem • While graph has 2 vertices – Pick an edge uv at random – Contract it • End While • Output remaining edges “Not too small” “Random Handful” “Throw it away” • Fix some min cut. Say it has k edges. • If algorithm doesn’t contract any edge in this cut, then the algorithm outputs this cut – When contracting edge uv, both u & v are on same side of cut • So what is probability that this happens?

• Initially there are n vertices. • Claim 1: # edges in min cut=k every vertex has degree k total # edges nk/2 • Pr[random edge is in min cut] = # edges in min cut / total # edges k / (nk/2) = 2/n

• Now there are n-1 vertices. • Claim 2: min cut in remaining graph is k • Why? Every cut in remaining graph is also a cut in original graph. • So, Pr[ random edge is in min cut ] 2/(n-1)

• In general, when there are i vertices left Pr[ random edge is in min cut ] 2/i • So Pr[ alg never contracts an edge in min cut ]

Final Algorithm • Input: Graph G • Output: min cut, with probability 1/2 • For i=1, . . , n 2 • Start from G • While graph has 2 vertices – Pick an edge uv at random – Contract it • End While • Let Ei = { remaining edges } • End For • Output smallest Ei • Running time: O(m¢n 2) (m = # edges, n = # vertices) • With more beautiful ideas, improves to O(n 2)

How Many Min Cuts? • Our analysis: for any particular min cut, Pr[ algorithm finds that min cut ] 1/n 2 • So suppose C 1, C 2, . . . , Ct are all the min cuts • 1/n 2 fraction of the time the algorithm finds C 1 • 1/n 2 fraction of the time the algorithm finds C 2 • . . . • 1/n 2 fraction of the time the algorithm finds Ct • This is only possible if t · n 2 • Corollary: Any connected graph on n vertices has · n 2 min cuts. (Actually, min cuts)

How Many Min Cuts? • Corollary: Any connected graph on n vertices has min cuts. • Is this optimal? • An n-cycle has exactly min cuts!

How Many Approximate Min Cuts? • A similar analysis gives a nice generalization: • Theorem (Karger-Stein ‘ 96): Let G be a graph with min cut size k. Then # { cuts with k edges } n 2. • No other proof of this theorem is known!

Classes CS 466, CS 761 (still active? ), C&O 738 Books Mitzenmacher-Upfal Motwani-Raghavan Alon-Spencer