Algorithms v Readings SG Ch 2 3 v

Algorithms v Readings: [SG] Ch. 2 & 3 v Chapter Outline: 1. Chapter Goals 2. What are Algorithms 1. Real Life Examples (origami, recipes) 2. Definition 3. Example: A = B + C 3. 4. 5. 6. Expressing Algorithms – Pseudo-Code Simple Algorithms Recursive Algorithms Time Complexity of Algorithms (UIT 2201: Algorithms) Page 1 Leong. HW, So. C, NUS

Algorithms v Computing devices are dumb o How to explain to a dumb mechanical / computing device how to solve a problem v How to solve a problem using “a small, basic set of primitive instructions”. v Complexity of Solving Problems. (UIT 2201: Algorithms) Page 2 Leong. HW, So. C, NUS

1. Goals of Algorithm Study v To develop framework for instructing computer to perform tasks v To introduce notion of algorithm as means of specifying how to solve a problem v To introduce and appreciate approaches for defining and solving very complex tasks in terms of simpler tasks; (UIT 2201: Algorithms) Page 3 Leong. HW, So. C, NUS

v Chapter Outline: 1. Chapter Goals 2. What are Algorithms 1. Real Life Examples (origami, recipes) 2. Definition of Algorithm 3. Example: A = B + C 3. 4. 5. 6. Expressing Algorithms – Pseudo-Code Simple Algorithms Recursive Algorithms Time Complexity of Algorithms (UIT 2201: Algorithms) Page 4 Leong. HW, So. C, NUS

2. Computer Science and Algorithms… v Computer Science can be considered… as study of algorithms including utheir formal properties u. Their hardware and software realisations u. Their applications to diverse areas (UIT 2201: Algorithms) Page 5 Leong. HW, So. C, NUS

Algorithms: Real Life Examples v Many Real-Life Examples o Cooking: Recipe for preparing a dish o Origami: The Art of Paper Folding o Directions: How to go to Changi Airport v Remember: o Framework: How to give instructions; o Alg: The actual step-by-step instructions; o Abstraction: Decomposing / Simplifying (UIT 2201: Algorithms) Page 6 Leong. HW, So. C, NUS

Real Life Example: Cooking v Framework: “Cooking or Recipe language” v Algorithm: “Recipe for a Dish” (step by step instructions on how to cook a dish) v Problem Decomposition o Preparing Ingredients; o Preparing Sauce; (UIT 2201: Algorithms) Page 7 Leong. HW, So. C, NUS

Real Life Example: Origami v Framework: “Origami or Paper-Folding language” v Algorithm: “Sequence of Paper-Folding Instructions” (step by step instructions for each fold) v Problem Decomposition o Start with a Bird Base; … o Finish the Head; o Finish the Legs; Finish the Tail; (UIT 2201: Algorithms) Page 8 Leong. HW, So. C, NUS

Real Life Examples: Issues v Problems/Difficulties: o Imprecise Instructions; o Job can often be done even if instructions are not followed precisely o Modifications may be done by the person following the instructions; v But, NOT for a Computer o Needs to told PRECISELY what to do; u. Instructions must be PRECISE; u. Cannot be vague or ambiguous (UIT 2201: Algorithms) Page 9 Leong. HW, So. C, NUS

Definition of Algorithm: v An algorithm for solving a problem “a finite sequence of unambiguous, executable steps or instructions, which, if followed would ultimately terminate and give the solution of the problem”. v Note the keywords: o o Finite set of steps; Unambiguous; Executable; Terminates; (UIT 2201: Algorithms) Page 10 Leong. HW, So. C, NUS

Algorithm Examples? v Problem 1: What is the largest integer INPUT: All the integers { … -2, -1, 0, 1, 2, … } OUTPUT: The largest integer Algorithm: u Arrange all the integers in a list in decreasing order; u MAX = first number in the list; u Print out MAX; o WHY is the above NOT an Algorithm? u (Hint: How many integers are there? ) v Problem 2: Who is the tallest women in the world? o Algorithm: Tutorial. . . (UIT 2201: Algorithms) Page 11 Leong. HW, So. C, NUS

Example: Adding two (n-digit) numbers o Input: Two positive m-digit decimal numbers (a and b) am, am-1, …. , a 1 bm, bm-1, …. , b 1 o Output: The sum c = a + b cm+1, cm-1, …. , c 1 (Note: In the textbook, it was am-1, …, a 1, a 0 …) (UIT 2201: Algorithms) Page 12 Leong. HW, So. C, NUS

How to “derive” the algorithm v Adding is something we all know o done it a thousand times, know it “by heart” v How do we give the algorithm? o A step-by-step instruction o to a dumb machine v Try an example: 3492 8157 “Imagine you looking at yourself solving it” (UIT 2201: Algorithms) Page 13 Leong. HW, So. C, NUS

Algorithm: Finding sum of A & B Step 1: Set the value of carry to 0 Step 2: Set the value of i to 1. Step 3: Repeat steps 4 through 6 until the value of i is > m. Step 4: Add ai and bi to the current value of carry, to get xi. Step 5: If xi < 10 then let ci=xi and reset carry to 0. Else (* i. e. xi 10 *) let ci=xi - 10 and reset carry to 1. Step 6: Increase the value of i by 1. Step 7: Set cm+1 to the value of carry. Step 8: Print the final answer cm+1, cm, …. , c 1 Step 9: Stop. (UIT 2201: Algorithms) Page 14 Leong. HW, So. C, NUS

v Chapter Outline: 1. Chapter Goals 2. What are Algorithms 3. Expressing Algorithms – Pseudo-Code 1. 2. 3. 4. 5. Communicating Alg to computer Pseudo-Code Primitive Operations and examples Variables and Arrays Algorithm C=A+B in pseudo-code 4. Simple Algorithms 5. Recursive Algorithms 6. Time Complexity of Algorithms (UIT 2201: Algorithms) Page 15 Leong. HW, So. C, NUS

3. Expressing Algorithms to Computer v To communicate algorithm to computer o Need way to “represent” the algorithm o Cannot use English v Can use computer language o machine language and o programming languages (Java, Pascal, C) o But, these are too tedious (&technical) v Use Pseudo-Code instead… (UIT 2201: Algorithms) Page 16 Leong. HW, So. C, NUS

Pseudo-Code to express Algorithms v Pseudo-Code o Mixture of computer language and English u. Somewhere in between uprecise enough to describe what is meant without being too tediuos o Examples: u Let c be 0; u Sort the list of numbers in increasing order; v Need to know both o syntax – representation o semantics – meaning (UIT 2201: Algorithms) Page 17 Leong. HW, So. C, NUS

Primitive Operations v To describe an algorithm, we need some well-defined programming primitives o Assignment primitive: u assignment statement, input/output statements o Conditional primitive: u if statement u case statement o Looping (iterative) primitive: u for loop, u while loop, v Statements are executed one-by-one (UIT 2201: Algorithms) Page 18 Leong. HW, So. C, NUS

Conditional Primitives… v if statement o to take different actions based on condition v Syntax if (condition) then (Step A) else (Step B) endif true Step A condition? false Step B if (condition) then (Step A) endif v Semantics (UIT 2201: Algorithms) Page 19 Leong. HW, So. C, NUS

Examples -- (if statement)… Let mark be the total-mark obtained if (mark < 40) then (print “Student fail”) else (print “Student pass”) endif … read in mark (*from the terminal*) if (mark < 40) then (Grade “F”) else if (mark < 50) then (Grade else if (mark < 60) then (Grade else if (mark < 70) then (Grade else if (mark < 80) then (Grade endif print “Student grade is”, Grade “D”) “C”) “B”) “A”); … (UIT 2201: Algorithms) Page 20 Leong. HW, So. C, NUS

Looping Primitive – while-loop v the while-loop o loop a “variable” number of times v Syntax while (condition) do (some sequence of statements) condition? false true Some sequence of statements; endwhile v Semantics… (UIT 2201: Algorithms) Page 21 Leong. HW, So. C, NUS

Looping Primitive – for-loop v First, the for-loop o loop a “fixed” or (predetermined) number of times v Syntax for j a to b do (some sequence of statements) j a; (j <= b)? false true Some sequence of statements; endfor v Semantics… j j+1; (UIT 2201: Algorithms) Page 22 Leong. HW, So. C, NUS

“Exercising the alg”: for and while for j 1 to 4 do print 2*j; endfor print “--- Done ---” Output: 2 4 6 8 --- Done --- j 1; while (j <= 4) do print 2*j; j j + 1; endwhile print “--- Done ---” Output: 2 4 6 8 --- Done --(UIT 2201: Algorithms) Page 23 Leong. HW, So. C, NUS

Variables and Arrays… v In the computer, each variable is assigned a storage “box” o can store one number at any time o eg: sum, j, carry v Arrays: o Often deal with many numbers o Such as A 1, A 2, A 3, … , A 100 o Store as an “array” A[1], A[2], … , A[100] uwe treat each of them as a variable, ueach is assigned a storage “box” (UIT 2201: Algorithms) Page 24 Leong. HW, So. C, NUS

Algorithm: A = B + C (in pseudo-code) We can re-write the C=A+B algorithm as follows: Alg. to Compute C = A + B: (*sum two big numbers*) carry 0; for i 1 to m do x[i] a[i] + b[i] + carry ; if (x[i] < 10) then ( c[i] x[i]; carry 0; ) else ( c[i] x[i] – 10; carry 1; ) endfor; c[m+1] carry; Print c[m+1], c[m], …. , c[1] (UIT 2201: Algorithms) Page 25 Leong. HW, So. C, NUS

v Chapter Outline: 1. 2. 3. 4. Chapter Goals What are Algorithms Expressing Algorithms – Pseudo-Code Simple Algorithms 1. Simple iterative algorithms Computing Sum, Find Max/Min 2. Modular Program Design 3. Divisibility and Prime Numbers 5. Recursive Algorithms 6. Time Complexity of Algorithms (UIT 2201: Algorithms) Page 26 Leong. HW, So. C, NUS

Simple iterative algorithm: Sum v Given: List of numbers: A 1, A 2, A 3, …. , An v Output: To compute the sum of the numbers Note: Store numbers in array A[1], A[2], … , A[n] Sum(A, n); begin Sum_sf 0; k 1; while (k <= n) do Sum_sf + A[k]; k k + 1; endwhile Sum_sf; Print “Sum is”, Sum end; (UIT 2201: Algorithms) Page 27 Leong. HW, So. C, NUS

Exercising Algorithm Sum: Input: A[1] A[2] A[3] A[4] A[5] A[6] 2 5 10 3 12 24 Processing: Output: k ? 1 2 3 4 5 6 6 Sum-sf 0 2 7 17 20 32 56 56 n=6 Sum ? ? ? ? 56 Sum is 56 (UIT 2201: Algorithms) Page 28 Leong. HW, So. C, NUS

Algorithm for Sum (with for-loop) v We can also use a while-loop instead of a for loop. Sum(A, n); (* Find the sum of A 1, A 2, …, An. *) begin Sum_sf 0; for k 1 to n do Sum_sf + A[k]; endfor Sum_sf; Print “Sum is”, Sum end; v HW: (a) Note the differences… (b) Modify it to compute the average? (UIT 2201: Algorithms) Page 29 Leong. HW, So. C, NUS

Remarks about the iterative algorithm… v Note three stages: 1. Initialization u Set some values at the beginning 2. Iteration u This is the KEY STEP u Where most of work is done 3. Post-Processing or Cleanup v Can use this setup for other problems o Calculating average, sum-of-squares o Finding max, min; Searching for a number, (UIT 2201: Algorithms) Page 30 Leong. HW, So. C, NUS

Modular Program Design v Software complex o HUGE (millions of lines of code) eg: Linux, Outlook o COMPLEX; eg: Flight simulator v Idea: Divide-and-Conquer Method o Complex tasks can be divided and each part solved separately and combined later. v Modular Program Design o Divide big programs into smaller modules o The smaller parts are ucalled modules, subroutines, or procedures u. Design, implement, and test separately o Modularity, Abstraction, Division of Labour o Simplifies process of writing alg/programs (UIT 2201: Algorithms) Page 31 Leong. HW, So. C, NUS

Simple Algorithms: Prime Numbers v Algorithm to determine if n is prime Given: A positive integer n Question: Is n a prime number? v What do we know? “A number n is prime iff n has no positive factors except 1 and n” v Idea: Express it “algorithmically” “n is not divisible by any positive number k such that 1 < k < n. ” or k=2, 3, 4, …, (n-1) v What we already have: o A module Divisible(m, n) to check divisibility o We can use it as a primitive (UIT 2201: Algorithms) Page 32 Leong. HW, So. C, NUS

Pseudo-Code for Prime: Prime(n) (* To determine if n is prime *) begin for k 2 to (n-1) do if Divisible(k, n) then (Print “False” & exit) else (* Do nothing *) endfor Print “True” & Stop end; v Note: Prime uses the module Divisible(k, n) v Exercise it with: o Prime (5); Prime (4); o Prime (55); Prime (41); (UIT 2201: Algorithms) Page 33 Leong. HW, So. C, NUS

A Simple Module: Divisibility v. A module (procedure) for divisibility Given: Two positive integers m and n Question: Is n divisible by m? or Algorithm to compute Divisible(m, n) v Algorithm Idea: “A positive integer n is divisible by m iff for some positive integer k n, m*k = n. ” (UIT 2201: Algorithms) Page 34 Leong. HW, So. C, NUS

Module for Divisibility: v Algorithm (in pseudo-code) v Exercise it with: o Divisible (3, 9); Divisible (3, 5); Divisible (3, 101); Divisible(m, n) (* To compute if n is divisible by m *) begin D false; (* assume false, first *) for k 1 to n do if (m*k = n) then (D true; exit-loop) else (* Do nothing *) endfor Divisible D; (* true or false *) end; (UIT 2201: Algorithms) Page 35 Leong. HW, So. C, NUS

v Chapter Outline: 1. 2. 3. 4. 5. 6. Chapter Goals What are Algorithms Expressing Algorithms – Pseudo-Code Simple Algorithms Recursive Algorithms Time Complexity of Algorithms 1. Sequential search algorithm 2. Binary search algorithm 3. Analysis of Time Complexity 7. Summary… (UIT 2201: Algorithms) Page 36 Leong. HW, So. C, NUS

Search: sequential search algorithm v Given: v List of numbers: A 1, A 2, A 3, …. , An and a query number x; Question: Search for x in the list; Sequential-Search(A, n, x); (* Search for x in A 1, A 2, …, An. *) begin for k 1 to n do if (x = A[k]) then (Print “Yes”; Stop) else (* do nothing *) endfor Print “No”; Stop end; (UIT 2201: Algorithms) Page 37 Leong. HW, So. C, NUS

Remarks on Sequential Search Alg v Analogy: House-to-house search… v How fast is it to search for x? o How many comparisons do we need? o Example: 13, 38, 19, 74, 76, 14, 12, 38, 22, 55 u. When x=14, need 6 comparisons u. When x=13, need only 1 comparison BEST CASE u. When x=55, need 10 comparisons WORST CASE u. When x=5, need 10 comparisons WORST CASE v In general, given n numbers, A 1, …, An o Best Case: 1 comparison o Worst Case: n comparisons o Average Case: (n+1)/2 comparisons (UIT 2201: Algorithms) Page 38 Leong. HW, So. C, NUS

Binary Search v If the List is sorted, that is A 1 A 2 A 3 …. An v Then, we can do better, o actually a lot better…. (UIT 2201: Algorithms) Page 39 Leong. HW, So. C, NUS

Binary Search 1, 4, 9, 11, 14, 43, 78 (UIT 2201: Algorithms) Page 40 Leong. HW, So. C, NUS

Binary Search (How fast is it? ) v Imagine o o o o n=100 After 1 step, size is 50 After 2 steps, size is 25 After 3 steps, size is 12 After 4 steps, size is 6 After 5 steps, size is 3 After 6 steps, size is 1 DONE!! v What if n=1000? 1, 000? (UIT 2201: Algorithms) Page 41 Leong. HW, So. C, NUS

Binary Search Algorithm Sorted List A 1 A 2 A 3 …. An A number x. Question: Is x in the list? Binary-Search(A, n, x); 1. First 1 Last n 2. While ( First Last ) do mid (first+last) / 2 If ( x = A[mid] ) then (output Yes and Stop) else If ( x < A[mid] ) then Last mid-1 else If ( x > A[mid] ) then First mid+1 End. While 3. Output False and Stop 4. End Input: (UIT 2201: Algorithms) Page 42 Leong. HW, So. C, NUS

Time Complexity Analysis v Sequential Search (Alg): o Worst Case: n comparisons o Best Case: 1 comparison o Avg Case: n/2 comparisons v Binary Search (Alg): o Worst Case: log 2 n comparisons o Best Case: 1 comparison o Avg Case: about log 2 n comparisons v How to get the Average Case? o using mathematical analysis… (UIT 2201: Algorithms) Page 43 Leong. HW, So. C, NUS

Complexity of Algorithm… v Logarithmic Time Algorithm o Binary Search v A Linear Time Algorithm o Algorithm Sum(A, n) -- O(n) time o Algorithm Sequential-Search(A, n, x) – O(n) time v A Quadratic Time Algorithm o Simple Median-Find (T 2 -Q 3) v An Exponential Time Algorithm o All-Subsets(A, n) (UIT 2201: Algorithms) Page 44 Leong. HW, So. C, NUS

v Chapter Outline: 1. 2. 3. 4. 5. Chapter Goals What are Algorithms Expressing Algorithms – Pseudo-Code Simple Algorithms Recursive Algorithms 1. Recursion – the idea 2. Fibonacci sequence 3. Tower of Hanoi 6. Time Complexity of Algorithms (UIT 2201: Algorithms) Page 45 Leong. HW, So. C, NUS

5. Recursion v A problem solving method of “decomposing bigger problems into smaller sub-problems that are identical to itself. ” v General Idea: o Solve simplest (smallest) cases DIRECTLY uusually these are very easy to solve o Solve bigger problems using smaller sub-problems uthat are identical to itself (but smaller and simpler) v Abstraction: o To solve a given problem, we first assume that we ALREADY know how to solve it for smaller instances!! v Dictionary definition: recursion see recursion (UIT 2201: Algorithms) Page 46 Leong. HW, So. C, NUS

Example: Fibonacci Numbers… v Definition of Fibonacci numbers 1. F 1 = 1, 2. F 2 = 1, 3. for n>2, Fn = Fn-1 + Fn-2 v Problem: Compute Fn for any n. v The above is a recursive definition. o Fn is computed in-terms of itself o actually, smaller copies of itself – Fn-1 and Fn-2 v Actually, Not difficult: F 3 = 1 + 1 = 2 F 4 = 2 + 1 = 3 F 5 = 3 + 2 = 5 F 6 = 5 + 3 = 8 F 7 = 8 + 5 = 13 F 8 = 13 + 8 = 21 F 9 = 21 + 13 = 34 F 10 = 34 + 21 = 55 F 11 = 55 + 34 = 89 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, … (UIT 2201: Algorithms) Page 47 Leong. HW, So. C, NUS

Fibonacci Numbers: Recursive alg Fibonacci(n) (* Recursive, SLOW *) begin if (n=1) or (n=2) then Fibonacci(n) 1 (*simple case*) else Fibonacci(n) Fibonacci(n-1) + Fibonacci(n-2) endif end; v The v It above is a recursive algorithm is simple to understand elegant! v But, very SLOW (UIT 2201: Algorithms) Page 48 Leong. HW, So. C, NUS

Recursive Fibonacci Alg -- Remarks v How slow is it? o Eg: To compute F(6)… F(6) F(5) F(4) F(3) F(2) J F(3) F(2) F(3) F(1) F(2) F(1) HW: Can we compute it faster? (UIT 2201: Algorithms) Page 49 Leong. HW, So. C, NUS

Example: Tower of Hanoi A B C A B Given: Three Pegs A, B and C Peg A initially has n disks, different size, stacked up, larger disks are below smaller disks Problem: to move the n disks to Peg C, subject to 1. Can move only one disk at a time 2. Smaller disk should be above larger disk 3. Can use other peg as intermediate (UIT 2201: Algorithms) Page 50 Leong. HW, So. C, NUS C

Tower of Hanoi v How to Solve: Strategy… o Generalize first: Consider n disks for all n 1 o Our example is only the case when n=4 v Look at small instances… o How about n=1 u Of course, just “Move disk 1 from A to C” o How about n=2? 1. “Move disk 1 from A to B” 2. “Move disk 2 from A to C” 3. “Move disk 1 from B to C” (UIT 2201: Algorithms) Page 51 Leong. HW, So. C, NUS

Tower of Hanoi (Solution!) v General Method: o First, move first (n-1) disks from A to B o Now, can move largest disk from A to C o Then, move first (n-1) disks from B to C v Try this method for n=3 1. “Move disk 1 from A to C” 2. “Move disk 2 from A to B” 3. “Move disk 1 from C to B” 4. “Move disk 3 from A to C” 5. “Move disk 1 from B to A” 6. “Move disk 1 from B to C” 7. “Move disk 1 from A to C” (UIT 2201: Algorithms) Page 52 Leong. HW, So. C, NUS

Algorithm for Towel of Hanoi (recursive) v Recursive Algorithm o when (n=1), we have simple case o Else (decompose problem and make recursive-calls) Hanoi(n, A, B, C); (* Move n disks from A to C via B *) begin if (n=1) then “Move top disk from A to C” else (* when n>1 *) Hanoi (n-1, A, C, B); “Move top disk from A to C” Hanoi (n-1, B, C, A); endif end; (UIT 2201: Algorithms) Page 53 Leong. HW, So. C, NUS

Characteristics of Algorithm v Correctness v Complexity --- time, space (memory), etc v Ease of understanding v Ease of coding v Ease of maintainence (UIT 2201: Algorithms) Page 54 Leong. HW, So. C, NUS

v If you are new to algorithms o read the textbook o try out the algorithms o do the exercises … The End … (UIT 2201: Algorithms) Page 55 Leong. HW, So. C, NUS