Algorithms and Complexity Bioinformatics Spring 2008 Hiram College

Algorithms and Complexity Bioinformatics Spring 2008 Hiram College Algorithm definition slides taken from CPSC 171, courtesy of Obertia Slotterbeck

What is an algorithm? • • An algorithm is a well-ordered collection of unambiguous and effectively computable operations that, when executed, • produces a result and • halts in a finite amount of time.

AN EXAMPLE OF A VERY SIMPLE ALGORITHM • 1. Wet your hair. • 2. Lather your hair. • 3. Rinse your hair. • 4. Stop. Observe: Operations need not be executed by a computer only by an entity capable of carrying out the operations listed. We assume that The algorithm begins executing at the top of the list of operations. The "Stop" can be omitted if we assume the last line is an implied "Stop" operation.

A well-ordered collection of operations The question that must be answered is: At any point in the execution of the algorithm, do you know what operation is to be performed next? Well-ordered operations: 1. Wet your hair. Not well-ordered operations: 2. Lather your hair. 1. Either wet your hair or lather your hair. 3. Rinse your hair. 2. Rinse your hair.

Choices are allowed: Well-ordered operations: 1. If your hair is dirty, then a. Wet your hair. b. Lather your hair. c. Rinse your hair. 2. Else a. Go to bed. Note: We will often omit the numbers and the letters and assume a "top-down" reading of the operations.

Unambiguous operations The question that must be answered is: Does the computing entity understand what the operation is to do? This implies that the knowledge of the computing entity must be considered. For example, is the following ambiguous? Make the pie crusts.

To an experienced cook, Make the pie crusts. is not ambiguous. But, an less experienced cook may need: Take 1 1/3 cups of flour. Sift the flour. Mix the sifted flour with 1/2 cup of butter and 1/4 cup of water to make dough. Roll the dough into two 9 -inch pie crusts. or even more detail!

Definition: An operation that is unambiguous is called a primitive operation (or just a primitive) One question we will be exploring in the course is what are the primitives of a computer. Note that a given collection of operations may be an algorithm with respect to one computing agent, but not with respect to another computing agent!!

Primitives for Computer Algorithms (e. g. PERL) • Mathematical operations: add, subtract, multiply, divide, log, sqrt, … • String operations: append, substring, reverse, … • File operations: read, write, append • Other I/O: print, scan

Primitives for Biological Algorithms • • Bind (a molecule binds to a site) Separate (strands of DNA) Polymerize (add base to strand) Repair gaps • These primitives make up an algorithm for DNA replication (A, pp. 14 -16)

Effectively computable operations The question that must be answered is: Is the computing entity capable of doing the operation? This assumes that the operation must first be unambiguousi. e. the computing agent understands what is to be done. Not effectively computable operations: Write all the fractions between 0 and 1. Create matter from nothing

that, when executed, produces a result The question that must be answered is: Can the user of the algorithm observe a result produced by the algorithm? The result need not be a number or piece of text viewed as "an answer". It could be an alarm, signaling something is wrong. It could be an approximation to an answer. It could be an error message.

halts in a finite amount of time The question that must be answered is: Will the computing entity complete the operations in a finite number of steps and stop? Do not confuse "not finite" with "very, very large". A failure to halt usually implies there is an infinite loop in the collection of operations: 1. Write the number 1 on a piece of paper. 2. Add 1 to the number you just wrote and write it on a piece of paper. 3. Repeat 2. 4. Stop.

Definition of an algorithm: • An algorithm is a well-ordered collection of unambiguous and effectively computable operations that, when executed, produces a result and halts in a finite amount of time. Note: Although I have tried to give clean cut examples to illustrate what these new words mean, in some cases, a collection of operations can fail for more than one reason.

A Language for Algorithms • Natural Language (English)? – Too ambiguous • Programming Language (Perl)? – Too much new syntax to learn • Pseudocode – A compromise. (… just right)

What is Pseudocode? • Structured like a programming language, but ignores many syntactical details (like $ and ; ) • Complex operations can be written in natural language • Still, we need to agree on some standard operations…

Our Pseudocode (pp. 8 -11) • • • Assignment Arithmetic Conditional execution Repeated execution Array access Functions

Assignment and Variables • A variable has a name (which can be anything in pseudocode) and a value. • Assignment changes the value • Examples: – my. Name <- “Ellen Walker” – number <- 17 – copy <- number – number <- 98765

Arithmetic • In pseudocode, mathematical symbols are allowed • But, programming language style math is easier to type dist <- sqrt((x[2]-x[1])^2 + (y[2]-y[1])^2)

Conditional (If Statement) • Allows a choice to be made, given: – a condition, – something to do if the condition is true – something to do if the condition is false. • Example: if today is a weekday go to work else stay home

Repeated Execution (Loops) • Need to know: – Which instructions to repeat – When to stop repeating • Two kinds of loops – While loop: stop when a condition is true – For loop: repeat a specific number of times

For Loop • Loop controlled by a variable • Executes once for each value of the variable, from a given starting value to a given ending value • Example: sum <- 0 for num <- 1 to 10 sum <- sum + num

While Loop • Loop is controlled by a condition. • When the condition is false, the loop no longer executes. • Example while (you are cold) raise thermostat temperature 1 degree

Array Access • An array is a sequence of values of the same type. • We access each item, or element by its numeric index. • Computers start counting at 0!

Array Example Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 0 2 3 4 5 7 10 11 1 6 8 9 Months <- {“Jan”, “Feb”, “Mar”, “Apr”, “May”, “Jun”, “Jul”, “Aug”, “Sep”, “Oct”, “Nov”, “Dec”} print(months[5]) for m<- 0 to 11 print(months[m]) go to the next line prints Jun

Assignment to an Array • An array element is really a variable, so you can assign to it. • Example: for n <- 0 to 99 squares[n] = n*n

Function • A function has a name, parameters (inputs), code to execute, and a return value. • Example: fibonacci (n) f[0] <- 1 f[1] <- 1 for i <- 2 to n-1 f[i] <- f[i-1]+f[i-2] return f[n-1]

Calling a Function • Write the function name, and the actual values for the parameters • When the function is complete, the return value replaces the name of the function in any expression. • Example: print(fibonacci(8)) prints 21

Exercise • Find a set of instructions, written in English, on the Internet, preferably for a nonmathematical task. • There should be at least 5 steps, and a condition or a loop, preferably both. • Rewrite the instructions in pseudocode. • Possibilities: – Instruction manuals – Government sites – Game descriptions

Comparing Algorithms • There can be many different algorithms to solve the same problem • Better algorithms… – Get the correct answer (if possible) – Get “better” answers (otherwise) – Use fewer resources (time and space)

Algorithm Complexity • Time – How long does the algorithm take? – Abstract, don’t want answer to depend on which machine! • Space – How much space (arrays, variables) does the algorithm need?

Time Complexity of an Algorithm What we want to do is relate 1. the amount of work performed by an algorithm 2. and the algorithm's input size by a fairly simple formula.

STEPS FOR DETERMING THE TIME COMPLEXITY OF AN ALGORITHM • 1. Determine how you will measure input size. Ex: – N items in a list – N x M table (with N rows and M columns) – Two numbers of length N • 2. Choose an operation (or perhaps two operations) to count as a gauge of the amount of work performed. Ex: – Comparisons – Swaps – Copies – Additions Normally we don't count operations in input/output.

STEPS FOR DETERMING THE TIME COMPLEXITY OF AN ALGORITHM • 3. Decide whether you wish to count operations in the – Best case? - the fewest possible operations – Worst case? - the most possible operations – Average case? • This is harder as it is not always clear what is meant by an "average case". Normally calculating this case requires some higher mathematics such as probability theory. • 4. For the algorithm and the chosen case (best, worst, average), express the count as a function of the input size of the problem. For example, we determine by counting, statements such as. . .

EXAMPLES: • For n items in a list, counting the operation swap, we find the algorithm performs 10 n + 5 swaps in the worst case. • For an n X m table, counting additions, we find the algorithm perform nm additions in the best case. • For two numbers of length n, there are 3 n + 20 multiplications in the best case.

STEPS FOR DETERMING THE TIME COMPLEXITY OF AN ALGORITHM 5. Given the formula that you have determined, decide the complexity class of the algorithm. What is the complexity class of an algorithm? Question: Is there really much difference between 3 n 5 n + 20 and 6 n -3 especially when n is large?

But, there is a huge difference, for n large, between n n 2 and n 3 So we try to classify algorithm into classes, based on their counts and simple formulas such as n, n 2, n 3, and others. Why does this matter? It is the complexity of an algorithm that most affects its running time--not the machine or its speed

The TRS-80 ORDER WINS OUT Main language support: BASIC - typically a slow running language For more details on TRS-80 see: http: //en. wikipedia. org/wiki/TRS-80 The CRAY-YMP Language used in example: FORTRAN- a fast running language For more details on CRAY-YMP see: http: //en. wikipedia. org/wiki/Cray_Y-MP

CRAY YMP with FORTRAN complexity is 3 n 3 TRS-80 with BASIC complexity is 19, 500, 000 n n is: 10 3 microsec 100 3 millisec 200 millisec 2 sec 1000 3 sec 20 sec 2500 50 sec 10000 49 min 3. 2 min 1000000 95 years 5. 4 hours

Trying to maintain an exact count for an operation isn't too useful. Thus, we group algorithms that have counts such as n 3 n + 20 1000 n - 12 0. 00001 n +2 together. We say algorithms with these type of counts are in the class (n) read as the class of theta-of-n or all algorithms of magnitude n or all order-n algorithms

Similarly, algorithms with counts such as n 2 + 3 n 1/2 n 2 + 4 n - 5 1000 n 2 + 2. 54 n +11 are in the class (n 2). Other typical classes are those with easy formulas in n such as 1 n 3 2 n lg n k = lg n if and only if 2 k = n

lg n k = lg n if and only if 2 k = n lg 4 = ? lg 8 = ? lg 16 = ? lg 10 = ? Note that all of these are base 2 logarithms. You don't use any logarithm table as we don't need exact values (except on integer powers of 2). Look at the curves showing the growth for algorithms in (1), (n 2), (n 3), (lg n), (n lg n), (2 n) These are the major ones we'll use.

Figure 3. 4 Work = cn for Various Values of c

Figure 3. 10 Work = cn 2 for Various Values of c

Figure 3. 11 A Comparison of n and n 2

Figure 3. 21 A Comparison of n and lg n

Figure 3. 21 A Comparison of n and lg n

Figure 3. 25 Comparisons of lg n, n, n 2 , and 2 n

Making Change • US Change Problem – Input: amount of money, M, in cents – Output: smallest number of coins (quarters, dimes, nickels, and pennies) that add up to M

US Change Algorithm While M>0 c <- value of largest coin with value <= M give c coin to customer M <- M-c • (See mathematical version, p. 19)

US Change Examples: • 77 c – Quarter (77 -25 = 52) – Quarter (52 -25 = 27) – Quarter (27 -25 = 2) – Penny (2 -1 = 1)

Same Algorithm using Mathematics • Calculate and give the max # of quarters (25 c): – Q = floor (M / 25) – Give customer Q quarters – M = M - 25*Q • Calculate and give the max # of dimes (10 c). • Calculate and give the max # of nickels (5 c). • Give the rest in pennies.

Generalizing the Algorithm • Suppose a non-US money system has d (different) coins of denominations: c[0], c[1], … c[d-1] • And, c[0] > c[1] > … c[d-1] • Then, we can generalize the algorithm:

Generalizing, continued for i <- 0 to d-1 Calculate and give the max number of this coin (c[i])

Generalized Algorithm is Not Correct! • c = {25, 20, 10, 5, 1} • M = 40 • Result is 3 coins, not 2

Brute Force: Correct but Slow N <- 1 While (not done) construct a combination of N coins if it adds up to M return it (done=true) else if no more combinations of N coins N <- N+1

Improving the Change Algorithm • Try combinations in a useful order – We’ve already done this, looking at fewer coins first. • Generate as few combinations as possible – Use knowledge to bound the combinations tried.