CSCE 411 Design and Analysis of Algorithms Set

CSCE 411 Design and Analysis of Algorithms Set 1: Introduction Slides by Prof. Jennifer Welch Spring 2014 CSCE 411, Spring 2013: Set 1 1

Mechanics n https: //parasol. tamu. edu/people/hcchung/teaching/411 s 14. html n Sources: n n n [CLRS] (textbook) The Design and Analysis of Algorithms, Anany Levitin The Algorithm Design Manual, Steven S. Skiena CSCE 411, Spring 2013: Set 1 2

Introduction n What is an algorithm? n n a step-by-step procedure to solve a problem every program is the instantiation of some algorithm http: //blog. kovyrin. net/wp-content/uploads/2006/05/algorithm_c. png CSCE 411, Spring 2013: Set 1 3

Sorting Example n Solves a general, well-specified problem n n Problem has specific instances n n given a sequence of n keys, a 1, …, an, as input, produce as output a reordering b 1, …, bn of the keys so that b 1 ≤ b 2 ≤ … ≤ b n. [Dopey, Happy, Grumpy] or [3, 5, 7, 1, 2, 3] Algorithm takes every possible instance and produces output with desired properties n insertion sort, quicksort, heapsort, … CSCE 411, Spring 2013: Set 1 4

Challenge n Hard to design algorithms that are n n correct efficient implementable Need to know about n n design and modeling techniques resources - don't reinvent the wheel CSCE 411, Spring 2013: Set 1 5

Correctness n How do you know an algorithm is correct? n produces the correct output on every input Since there are usually infinitely many inputs, it is not trivial n Saying "it's obvious" can be dangerous n n often one's intuition is tricked by one particular kind of input CSCE 411, Spring 2013: Set 1 6

Tour Finding Problem n Given a set of n points in the plane, what is the shortest tour that visits each point and returns to the beginning? n n application: robot arm that solders contact points on a circuit board; want to minimize movements of the robot arm How can you find it? CSCE 411, Spring 2013: Set 1 7

Finding a Tour: Nearest Neighbor n n start by visiting any point while not all points are visited n n choose unvisited point closest to last visited point and visit it return to first point CSCE 411, Spring 2013: Set 1 8

Nearest Neighbor Counterexample -21 -5 -1 0 1 CSCE 411, Spring 2013: Set 1 3 11 9

How to Prove Correctness? n There exist formal methods n n even automated tools more advanced than this course In this course we will primarily rely on more informal reasoning about correctness n Seeking counter-examples to proposed algorithms is important part of design process n CSCE 411, Spring 2013: Set 1 10

Efficiency n Software is always outstripping hardware n n need faster CPU, more memory for latest version of popular programs Given a problem: n n n what is an efficient algorithm? what is the most efficient algorithm? does there even exist an algorithm? CSCE 411, Spring 2013: Set 1 11

How to Measure Efficiency n Machine-independent way: n n analyze "pseudocode" version of algorithm assume idealized machine model n n "Big-Oh" notation n n one instruction takes one time unit order of magnitude as problem size increases Worst-case analyses n safe, often occurs most often, average case often just as bad CSCE 411, Spring 2013: Set 1 12

Faster Algorithm vs. Faster CPU n n n A faster algorithm running on a slower machine will always win for large enough instances Suppose algorithm S 1 sorts n keys in 2 n 2 instructions Suppose computer C 1 executes 1 billion instruc/sec n n n When n = 1 million, takes 2000 sec Suppose algorithm S 2 sorts n keys in 50 nlog 2 n instructions Suppose computer C 2 executes 10 million instruc/sec n When n = 1 million, takes 100 sec CSCE 411, Spring 2013: Set 1 13

Caveat No point in finding fastest algorithm for part of the program that is not the bottleneck n If program will only be run a few times, or time is not an issue (e. g. , run overnight), then no point in finding fastest algorithm n CSCE 411, Spring 2013: Set 1 14

Modeling the Real World n Cast your application in terms of well-studied abstract data structures Concrete Abstract arrangement, tour, ordering, sequence permutation cluster, collection, committee, group, packaging, selection subsets hierarchy, ancestor/descendants, taxonomy trees network, circuit, web, relationship graph sites, positions, locations points shapes, regions, boundaries polygons text, characters, patterns strings CSCE 411, Spring 2013: Set 1 15

Real-World Applications n n n Hardware design: VLSI chips Compilers Computer graphics: movies, video games Routing messages in the Internet Searching the Web Distributed file sharing n n n Computer aided design and manufacturing Security: e-commerce, voting machines Multimedia: CD player, DVD, MP 3, JPG, HDTV DNA sequencing, protein folding and many more! CSCE 411, Spring 2013: Set 1 16

Some Important Problem Types n Sorting n n among a set of items n text, bit strings, gene sequences n graphics, imaging, robotics Numerical n model objects and their relationships find desired permutation, combination or subset Geometric n Graphs n Combinatorial n String processing n n a set of items Searching n n n continuous math: solving equations, evaluating functions CSCE 411, Spring 2013: Set 1 17

Algorithm Design Techniques n Brute Force & Exhaustive Search n Dynamic Programming n break problem into overlapping subproblems follow definition / try all possibilities n Greedy n Divide & Conquer n repeatedly do what is best now n break problem into distinct subproblems n Iterative Improvement n n Transformation n convert problem to another one n n repeatedly improve current solution Randomization n use random numbers CSCE 411, Spring 2013: Set 1 18

Plan of the Course n Cover a variety of fundamental algorithm design and analysis techniques as applied to a number of basic problems n n In the other direction, study some lower bounds, indicating inherent limitations for finding efficient algorithms n n Organize by technique including NP-completeness Learn about undecidability: some problems are unsolvable CSCE 411, Spring 2013: Set 1 19