CS 3343 Analysis of Algorithms Lecture 4 Sum
- Slides: 63
CS 3343: Analysis of Algorithms Lecture 4: Sum of series, Analyzing recursive algorithms 11/25/2020 1
Outline • Review of last lecture • Sum of series • Analyzing recursive algorithms 11/25/2020 2
L’ Hopital’s rule lim f(n) / g(n) = lim f(n)’ / g(n)’ n→∞ 11/25/2020 n→∞ Condition: If both lim f(n) and lim g(n) = or 0 3
Stirling’s formula or (constant) 11/25/2020 4
Properties of asymptotic notations • Textbook page 51 • Transitivity f(n) = (g(n)) and g(n) = (h(n)) => f(n) = (h(n)) (holds true for o, O, , and as well). • Symmetry f(n) = (g(n)) if and only if g(n) = (f(n)) • Transpose symmetry f(n) = O(g(n)) if and only if g(n) = (f(n)) f(n) = o(g(n)) if and only if g(n) = (f(n)) 11/25/2020 5
logarithms • • lg n = log 2 n ln n = loge n, e ≈ 2. 718 lgkn = (lg n)k lg lg n = lg (lg n) = lg(2)n lg(k) n = lg lg lg … lg n lg 24 = ? lg(2)4 = ? Compare lgkn vs lg(k)n? 11/25/2020 6
Useful rules for logarithms • For all a > 0, b > 0, c > 0, the following rules hold • logba = logca / logcb = lg a / lg b • logban = n logba = a b • • log (ab) = log a + log b – lg (2 n) = ? • log (a/b) = log (a) – log(b) – lg (n/2) = ? – lg (1/n) = ? • logba = 1 / logab 11/25/2020 7
Useful rules for exponentials • For all a > 0, b > 0, c > 0, the following rules hold • a 0 = 1 (00 = ? ) • a 1 = a • a-1 = 1/a • (am)n = amn • (am)n = (an)m • aman = am+n 11/25/2020 8
More advanced dominance ranking 11/25/2020 9
General plan for analyzing time efficiency of a non-recursive algorithm • Decide parameter (input size) • Identify most executed line (basic operation) • worst-case = average-case? • T(n) = i ti • T(n) = Θ (f(n)) 11/25/2020 10
Find the order of growth for sums • • • T(n) = i=1. . n i = Θ (n 2) T(n) = i=1. . n log (i) = ? T(n) = i=1. . n n / 2 i = ? T(n) = i=1. . n 2 i = ? … • How to find out the actual order of growth? – Math… – Textbook Appendix A. 1 (page 1058 -60) 11/25/2020 11
Arithmetic series • An arithmetic series is a sequence of numbers such that the difference of any two successive members of the sequence is a constant. e. g. : 1, 2, 3, 4, 5 or 10, 12, 14, 16, 18, 20 • In general: Recursive definition Or: 11/25/2020 Closed form, or explicit formula 12
Sum of arithmetic series If a 1, a 2, …, an is an arithmetic series, then e. g. 1 + 3 + 5 + 7 + … + 99 = ? (series definition: ai = 2 i-1) This is ∑ i = 1 to 50 (ai) 11/25/2020 13
Geometric series • A geometric series is a sequence of numbers such that the ratio between any two successive members of the sequence is a constant. e. g. : 1, 2, 4, 8, 16, 32 or 10, 20, 40, 80, 160 or 1, ½, ¼, 1/8, 1/16 • In general: Recursive definition Or: 11/25/2020 Closed form, or explicit formula 14
Sum of geometric series if r < 1 if r > 1 if r = 1 11/25/2020 15
Sum of geometric series if r < 1 if r > 1 if r = 1 11/25/2020 16
Important formulas 11/25/2020 17
Sum manipulation rules Example: 11/25/2020 18
Sum manipulation rules Example: 11/25/2020 19
Examples • i=1. . n n / 2 i = n * i=1. . n (½)i = ? • using the formula for geometric series: i=0. . n (½)i = 1 + ½ + ¼ + … (½)n = 2 • Application: algorithm for allocating dynamic memories 11/25/2020 20
Examples • i=1. . n log (i) = log 1 + log 2 + … + log n = log 1 x 2 x 3 x … x n = log n! = (n log n) • Application: algorithm for selection sort using priority queue 11/25/2020 21
Problem of the day How do you find a coffee shop if you don’t know on which direction it might be? 11/25/2020 22
Recursive definition of sum of series • T (n) = i=0. . n i is equivalent to: Recurrence relation T(n) = T(n-1) + n Boundary condition T(0) = 0 • T(n) = i=0. . n ai is equivalent to: T(n) = T(n-1) + an T(0) = 1 Recursive definition is often intuitive and easy to obtain. It is very useful in analyzing recursive algorithms, and some non-recursive algorithms too. 11/25/2020 23
Analyzing recursive algorithms 11/25/2020 24
Recursive algorithms • General idea: – Divide a large problem into smaller ones • By a constant ratio • By a constant or some variable – Solve each smaller one recursively or explicitly – Combine the solutions of smaller ones to form a solution for the original problem Divide and Conquer 11/25/2020 25
Merge sort MERGE-SORT A[1. . n] 1. If n = 1, done. 2. Recursively sort A[ 1. . n/2 ] and A[ n/2 +1. . n ]. 3. “Merge” the 2 sorted lists. Key subroutine: MERGE 11/25/2020 26
Merging two sorted arrays Subarray 1 Subarray 2 20 12 13 11 11/25/2020 7 9 2 1 27
Merging two sorted arrays Subarray 1 Subarray 2 20 12 13 11 11/25/2020 7 9 2 1 28
Merging two sorted arrays 20 12 13 11 7 9 2 1 11/25/2020 29
Merging two sorted arrays 20 12 13 11 7 9 2 1 11/25/2020 30
Merging two sorted arrays 20 12 13 11 7 9 2 1 1 11/25/2020 31
Merging two sorted arrays 20 12 13 11 7 9 7 2 1 2 9 1 11/25/2020 32
Merging two sorted arrays 20 12 13 11 7 9 7 2 1 11/25/2020 9 2 33
Merging two sorted arrays 20 12 13 11 7 9 7 7 2 1 11/25/2020 9 9 2 34
Merging two sorted arrays 20 12 13 11 7 9 7 7 2 1 11/25/2020 9 2 9 7 35
Merging two sorted arrays 20 12 13 11 7 9 7 7 2 1 11/25/2020 9 2 9 9 7 36
Merging two sorted arrays 20 12 13 11 7 9 7 7 2 1 11/25/2020 9 2 9 7 9 9 37
Merging two sorted arrays 20 12 20 12 13 11 13 11 7 9 7 7 2 1 11/25/2020 9 2 9 7 9 9 38
Merging two sorted arrays 20 12 20 12 13 11 13 11 7 9 7 7 2 1 11/25/2020 9 2 9 7 9 9 11 39
Merging two sorted arrays 20 12 20 12 13 11 13 11 13 7 9 7 7 2 1 11/25/2020 9 2 9 7 9 9 11 40
Merging two sorted arrays 20 12 20 12 13 11 13 11 13 7 9 7 7 2 1 11/25/2020 9 2 9 7 9 9 11 12 41
How to show the correctness of a recursive algorithm? • By induction: – Base case: prove it works for small examples – Inductive hypothesis: assume the solution is correct for all sub-problems – Step: show that, if the inductive hypothesis is correct, then the algorithm is correct for the original problem. 11/25/2020 42
Correctness of merge sort MERGE-SORT A[1. . n] 1. If n = 1, done. 2. Recursively sort A[ 1. . n/2 ] and A[ n/2 +1. . n ]. 3. “Merge” the 2 sorted lists. Proof: 1. 2. 3. 11/25/2020 Base case: if n = 1, the algorithm will return the correct answer because A[1. . 1] is already sorted. Inductive hypothesis: assume that the algorithm correctly sorts A[1. . n/2 ] and A[ n/2 +1. . n]. Step: if A[1. . n/2 ] and A[ n/2 +1. . n] are both correctly sorted, the whole array A[1. . n/2 ] and A[ n/2 +1. . n] is sorted after merging. 43
How to analyze the time-efficiency of a recursive algorithm? • Express the running time on input of size n as a function of the running time on smaller problems 11/25/2020 44
Analyzing merge sort T(n) MERGE-SORT A[1. . n] Θ(1) 1. If n = 1, done. 2 T(n/2) 2. Recursively sort A[ 1. . n/2 ] and A[ n/2 +1. . n ]. f(n) 3. “Merge” the 2 sorted lists Sloppiness: Should be T( n/2 ) + T( n/2 ) , but it turns out not to matter asymptotically. 11/25/2020 45
Analyzing merge sort 1. Divide: Trivial. 2. Conquer: Recursively sort 2 subarrays. 3. Combine: Merge two sorted subarrays T(n) = 2 T(n/2) + f(n) +Θ(1) # subproblems subproblem size 11/25/2020 1. What is the time for the base case? 2. What is f(n)? 3. What is the growth order of T(n)? Dividing and Combining Constant 46
Merging two sorted arrays 20 12 20 12 13 11 13 11 13 7 9 7 7 2 1 9 2 9 7 9 9 11 12 Θ(n) time to merge a total of n elements (linear time). 11/25/2020 47
Recurrence for merge sort T(n) = Θ(1) if n = 1; 2 T(n/2) + Θ(n) if n > 1. • Later we shall often omit stating the base case when T(n) = (1) for sufficiently small n, but only when it has no effect on the asymptotic solution to the recurrence. • But what does T(n) solve to? I. e. , is it O(n) or O(n 2) or O(n 3) or …? 11/25/2020 48
Binary Search To find an element in a sorted array, we 1. Check the middle element 2. If ==, we’ve found it 3. else if less than wanted, search right half 4. else search left half Example: Find 9 3 11/25/2020 5 7 8 9 12 15 49
Binary Search To find an element in a sorted array, we 1. Check the middle element 2. If ==, we’ve found it 3. else if less than wanted, search right half 4. else search left half Example: Find 9 3 11/25/2020 5 7 8 9 12 15 50
Binary Search To find an element in a sorted array, we 1. Check the middle element 2. If ==, we’ve found it 3. else if less than wanted, search right half 4. else search left half Example: Find 9 3 11/25/2020 5 7 8 9 12 15 51
Binary Search To find an element in a sorted array, we 1. Check the middle element 2. If ==, we’ve found it 3. else if less than wanted, search right half 4. else search left half Example: Find 9 3 11/25/2020 5 7 8 9 12 15 52
Binary Search To find an element in a sorted array, we 1. Check the middle element 2. If ==, we’ve found it 3. else if less than wanted, search right half 4. else search left half Example: Find 9 3 11/25/2020 5 7 8 9 12 15 53
Binary Search To find an element in a sorted array, we 1. Check the middle element 2. If ==, we’ve found it 3. else if less than wanted, search right half 4. else search left half Example: Find 9 3 11/25/2020 5 7 8 9 12 15 54
Binary Search Binary. Search (A[1. . N], value) { if (N == 0) return -1; // not found mid = (1+N)/2; if (A[mid] == value) return mid; // found else if (A[mid] < value) return Binary. Search (A[mid+1, N], value) else return Binary. Search (A[1. . mid-1], value); } What’s the recurrence relation for its running time? 11/25/2020 55
Recurrence for binary search 11/25/2020 56
Recursive Insertion Sort Recursive. Insertion. Sort(A[1. . n]) 1. if (n == 1) do nothing; 2. Recursive. Insertion. Sort(A[1. . n-1]); 3. Find index i in A such that A[i] <= A[n] < A[i+1]; 4. Insert A[n] after A[i]; 11/25/2020 57
Recurrence for insertion sort 11/25/2020 58
Compute factorial Factorial (n) if (n == 1) return 1; return n * Factorial (n-1); • Note: here we use n as the size of the input. However, usually for such algorithms we would use log(n), i. e. , the bits needed to represent n, as the input size. 11/25/2020 59
Recurrence for computing factorial • Note: here we use n as the size of the input. However, usually for such algorithms we would use log(n), i. e. , the bits needed to represent n, as the input size. 11/25/2020 60
What do these mean? Challenge: how to solve the recurrence to get a closed form, e. g. T(n) = Θ (n 2) or T(n) = Θ(nlgn), or at least some bound such as T(n) = O(n 2)? 11/25/2020 61
Solving recurrence • Running time of many algorithms can be expressed in one of the following two recursive forms or Both can be very hard to solve. We focus on relatively easy ones, which you will encounter frequently in many real algorithms (and exams…) 11/25/2020 62
Solving recurrence 1. Recursion tree / iteration method 2. Substitution method 3. Master method 11/25/2020 63
- Analysis of algorithms lecture notes
- Introduction to algorithms lecture notes
- 01:640:244 lecture notes - lecture 15: plat, idah, farad
- Int sum(int a int n) int sum=0 i
- 1001 design
- An introduction to the analysis of algorithms
- What is analysis of algorithm
- Association analysis: basic concepts and algorithms
- Input and output in algorithm
- Algorithm analysis examples
- Algorithm analysis examples
- Fundamentals of analysis of algorithm efficiency
- Cluster analysis basic concepts and algorithms
- Probabilistic analysis and randomized algorithms
- Design and analysis of algorithms introduction
- Cluster analysis basic concepts and algorithms
- Cjih
- Goals of analysis of algorithms
- Exercise 24
- Binary search in design and analysis of algorithms
- Introduction to the design and analysis of algorithms
- Competitive analysis algorithms
- Design and analysis of algorithms
- Design and analysis of algorithms
- Cluster analysis basic concepts and algorithms
- Comp 482
- Exploratory data analysis lecture notes
- Sensitivity analysis lecture notes
- Factor analysis lecture notes
- Streak plate method performed on
- Zline 667-36
- Computational thinking algorithms and programming
- Types of algorithm
- List of recursive algorithms
- Safe patient handling algorithms
- What is recursion
- Types of randomized algorithms
- Process mining algorithms
- Evolution of logistics ppt
- Nature-inspired learning algorithms
- Metaheuristic algorithms
- Making good encryption algorithms
- Statistical algorithms
- Ajit diwan iit bombay
- Greedy algorithm
- N/a greedy
- Consistent global state
- Classical algorithms for forrelation
- Aprioti
- Dsp programming tutorial
- Distributed algorithms nancy lynch
- Virtual web view
- Computer arithmetic: algorithms and hardware designs
- Princeton data structures and algorithms
- Routing algorithms in computer networks
- Data structures and algorithms tutorial
- Lossless compression algorithms in multimedia
- Memory management algorithms
- Basic raster graphics algorithm for 2d primitives
- Non recursive algorithm examples
- Bioalgorithms
- What is backtracking?
- Algorithms for select and join operations
- Algorithms and flowcharts