Efficiency of Algorithms Logarithmic Orders Binary Search Algorithm

Efficiency of Algorithms: Logarithmic Orders Binary Search Algorithm 1

Logarithmic function • Definition: The logarithmic function with base b (b>0, b 1) is the following function from R+ to R: logb(x) = the exponent to which b must raised to obtain x. Symbolically, logbx = y by = x. • Property: If the base b>1, then the logarithmic function is increasing: if x 1<x 2 , then logb(x 1) < logb(x 2). Note: Logarithmic function grows very slowly, e. g. , log 2(1, 024)=10, log 2(1, 048, 576)=20. 2

A property of logarithmic function • Proposition 1: If k is an integer and x is a real number with 2 k x < 2 k+1, then log 2 x = k. • Proof: 2 k x < 2 k+1 log 2(2 k) log 2(x) < log 2(2 k+1) (since log 2 x increasing) k log 2(x) < k+1 (by definition of log 2 x) log 2 x = k (by definition of floor function) ■

An application of logarithms • Question: Given a positive integer n, how many binary digits are needed to represent n? • Solution: Binary representation of n: 1 ck-2…c 2 c 1 c 0 which corresponds to n = 2 k + ck-1∙ 2 k-1 + … + c 2∙ 22 + c 1∙ 2 + c 0 Since ci 1, n = 2 k + ck-1∙ 2 k-1 + … + c 2∙ 22 + c 1∙ 2 + c 0 2 k + 2 k-1 + … + 22 + 1 = (2 k+1 -1) / (2 -1) (as a sum of geometric sequence) = 2 k+1 -1 < 2 k+1 (1) On the other hand, n = 2 k + ck-1∙ 2 k-1 + … + c 2∙ 22 + c 1∙ 2 + c 0 ≥ 2 k (2) Combining (1) and (2): 2 k n < 2 k+1 (3) log 2 n and the number of binary digits is log 2 n + 1. Based on (3) and Proposition 1, k = 4

Exponential and Logarithmic Orders Ø For all real numbers b and r with b>1 and r>0 and for all sufficiently large values of x, logbx xr; ( which implies that logbx is O(xr) ) xr b x ( which implies that xr is O(bx) ) Ø For all real numbers b with b>1 and for all sufficiently large values of x, x x logbx x 2 (which implies that x is O(x logbx) and x logbx is O(x 2) ) 5

Binary Search Algorithm • The algorithm searches for an element x in an ascending array of elements a[1], …, a[n]. • Algorithm body: index: =0, bot: =1, top: =n while (top ≥ bot and index=0) mid : = (bot+top) / 2 if a[mid] = x then index : = mid if a[mid] > x then top : = mid-1 else bot : = mid+1 end while Output: index (If index has the value 0 then x is not in the array; otherwise, index gives the index of the array where x is located. )

Binary Search Algorithm: Example • Suppose a[1]=Amy, a[2]=Bob, a[3]=Dave, a[4]=Erin, a[5]=Jeff, a[6]=John, a[7]=Larry, a[8]=Mary, a[9]=Mike, a[10]=Sam, a[11]=Steve, a[12]=Tom. (sorted in alphabetical order) • Search for x=Erin. • The table tracing the binary search algorithm: index bot 0 1 1 4 top 12 5 5 6 3 mid 4 4 7

The Efficiency of the Binary Search Algorithm • At each iteration, the length of the new subarray to be searched is approximately half of the previous one. • If n = 2 k+m, where 0 m < 2 k, then n can be split approximately in half k times. • Since 2 k n < 2 k+1, then k = log 2 n (by proposition 1) • Thus, the number of iterations of the while loop in a worst-case execution of the algorithm is log 2 n +1. • The number of operations in each loop is constant (doesn’t increase with n). • Thus, the binary search algorithm is O(log 2 n). 8