Fundamentals of Algorithms MCS2 Lecture 1 Introduction Prerequisites

Fundamentals of Algorithms MCS-2 Lecture 1 Introduction

Prerequisites • Data Structures • Discrete Mathematics Assessment & Evaluation Criteria • • • Course Resources Quizzes Assignments Mid Term Presentation Final Term • Total: 10 % 25 % 10 % 45 % 100 % Compulsory Reading Material • Introduction to Algorithms by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest & Clifford Stein (3 rd Edition) Recommended Reading • Fundamentals of Computer Algorithms by Eillies Horowitz & Sahni Sartaj (1990) • Fundamentals of Algorithms by Gilles Brassard & Paul Bratley (1995) • Algorithms; Fourth Edition by Robert Sedgewick & Kevin Wayne.

Problem Solving • Programming is a process of problem solving • Problem solving techniques o Analyze the problem o Outline the problem requirements o Design steps (algorithm) to solve the problem • Algorithm o Step-by-step problem-solving process o Solution achieved in finite amount of time Definition of Algorithm • In simple terms, an algorithm is a series of instructions to solve a problem (complete a task) • Problems can be in any form o Business • Allocate manpower to maximize profit

Definition o Real Life • If I go 10 steps forward, I will reach my mailbox. • I am hungry. How do I order pizza? • Explain how to tie shoelaces to a five year old child Application of Algorithms in CS Areas • • • Compiler construction Operating system Databases AI Graphics Networking

Algorithm & Programming • The idea behind the computer program • It is a mathematical entity of a program. • Stays the same independent of o Which kind of hardware it is running on o Which programming language it is written in • Solves a well-specified problem in a general way • Formally: “An algorithm is an ordered set of unambiguous executable steps, defining a terminating process. ” o Ordered set of steps: structure! o Executable steps: doable! o Unambiguous steps: follow the directions! o Terminating: must have an end!

Properties of Algorithms / Criterion for analyzing algorithms • Correct o always returns the desired output for all legal instances of the problem. • Unambiguous • Precise • Efficient o Can be measured in terms of • Running Time • Memory Space o Sometimes tends to be more important

Representation of Algorithms are generally represented by either • Verbal o The algorithm is expressed in words, sentences and paragraphs. o Usually verbose, long, and often inaccurate. • Algebraic o The algorithm is expressed mathematically with symbols and formulas. o This is usually a very concise representation. • Tabular o The algorithm is represented by one or more rectangular grids (tables, arrays or matrices) with entries in the grids. o This method is useful for summarizing large selections. • Hierarchical The algorithm is presented as a break-out diagram. • Data Flow Diagram o The algorithm is shown as a set of boxes that show the actions to be performed.

Representation of Algorithms • Flow chart o The algorithm is represented in the form of a diagram with action boxes linked by lines. • Pseudo code o The algorithm is presented as a set of instructions written using a mixture of natural language and mathematical notation. Pseudo code • It is a mixture of o natural language and mathematical notation o independent of any programming language. • Reflects the fact “short-term communication” between members working on a specific project • Pseudo code can be written in very formal constructs. It can be used almost like a free-verse description of what the program needs to do. • What the input is? • What the output should be?

Rules for Pseudo code • No strict rules exist. • Write only one statement per line • Capitalize initial keyword • Indent to show hierarchy • End multi-line structures • Keep statements language independent A Recommended Pseudo code Format • Documentation Keyword • Action Keywords • Flow Control Keywords

Documentation Keyword Documentation keywords describe what needs to be done or provides information about why something is being done. • TASK o TASK statement is something that the program must perform. • REM o REM statement is merely a remark or comment. Action Keywords Action keywords are the lines that actually do the work o Common Action Keywords o Input: READ, OBTAIN, GET o Output: PUT, PRINT, DISPLAY, SHOW o Compute: COMPUTE, CALCULATE, DETERMINE o Initialize: SET, INIT o Add one: INCREMENT, DECREMENT

Flow Control Keywords • Sequence Calculate Pay – sequence Sum of 2 Numbers – sequence Begin input hours input rate pay = hours * input x, y sum = x + y print sum End rate print pay End

Flow Control Keywords • Selection IF (test condition) Statement(s) to be executed if test condition is TRUE ELSE Statement(s) to be executed if test condition is FALSE If amount < 1000 interest. Rate =. 06 REM the “yes” or “true” action Else interest. Rate =. 10 REM the “no” or “false” action End If Calculate Pay with Overtime - selection Begin Input hours, rate If hours <= 40 then pay = hours * rate Else pay = 40 * rate + (hours – 40) * rate * 1. 5 Print pay End

Flow Control Keywords • Repetition - Case 1 • LOOP WHILE (test condition) Statement(s) to be executed if test condition is TRUE • Repetition - Case 2 • LOOP Statement(s) to be executed if test condition is TRUE Average of 10 Numbers – iteration with a for WHILE: (test condition) loop Begin sum = 0 For i = 1 To 10 Input x sum = sum + x avg = sum / 10. 0 Print avg End For End

Flow Control Keywords • Repetition - Case 3 • LOOP UNTIL (test condition) Statement(s) to be executed if test condition is FALSE • Repetition - Case 4 • LOOP Statement(s) to be executed if test condition is FALSE UNTIL (test condition)

Steps in development of Algorithms 1. 2. 3. 4. 5. 6. 7. 8. 9. Problem definition Development of a model Specification of Algorithm Designing an Algorithm Checking the correctness of Algorithm Analysis of Algorithm Implementation of Algorithm Program testing Documentation Preparation Algorithm Design • Algorithm design is a specific method to create a mathematical process in solving problems. Algorithm Design Techniques • Brute force • Greedy algorithms

Algorithm Design Techniques • Divide-and-conquer, decrease-and-conquer • Dynamic programming • Transform-and-conquer • Backtracking and branch-and-bound • Genetic algorithms Analysis of Algorithms • The goal of time complexity (computational complexity) is to classify algorithms according to their performances. • The time complexity of an algorithm quantifies the amount of time taken by an algorithm to run as a function of the length of the string representing the input. • Time complexity is commonly estimated by counting the number of elementary operations performed by the algorithm, where an elementary operation takes a fixed amount of time to perform. • Complexity is defined as a numerical function T(n) - time versus the input size n.

What is Function? • In mathematics, a function is a relation between a set of inputs and a set of permissible outputs with the property that each input is related to exactly one output. • For example a function that relates each real number x to its square x 2. The output of a function f corresponding to an input x is denoted by f(x) (read "f of x"). • The input to a function is called the argument and the output is called the value. The set of all permitted inputs to a given function is called the domain of the function, while the set of permissible outputs is called the codomain. • There are many ways to describe or represent a function. o Some functions may be defined by a formula or algorithm that tells how to compute the output for a given input. o Others are given by a picture, called the graph of the function. o In science, functions are sometimes defined by a table that gives the outputs for selected inputs.

How fast will your program run? • The running time of your program will depend upon: o The algorithm o The input o Your implementation of the algorithm in a programming language o The compiler you use o The OS on your computer o Your computer hardware o Maybe other things: temperature outside; other programs on your computer; … • Our Motivation To analyze the running time of an algorithm as a function of only simple parameters of the input. • Each algorithm performs a sequence of basic operations: o Arithmetic: (low + high)/2 o Comparison: if ( x > 0 ) … o Assignment: temp = x o Branching: while ( true ) { … } • Idea: To count number of basic operations performed on the input.

Asymptotic Notations • To judge how long two solutions will take to run, and choose the better of the two, you don't need to know how many minutes and seconds they will take, but you do need some way to compare algorithms against one another. • Asymptotic notation is a way of expressing the main component of the cost of an algorithm. • Its domain is the set of natural numbers N={0, 1, 2, …. } • Such notations are convenient for describing the worst-case running-time function T(n), which is usually defined only on integer input sizes. • Example o f(n) = Q(n 2). o Describes how f(n) grows in comparison to n 2. • Theta (Q) notation • Big O (O) notation o Little o (o) notation • Big Omega (W) notation o Little Omega (ω) notation

Asymptotic behavior / growth of functions • Constant time • Logn • Linear nlogn • • Quadratic • Cubic • Exponential • n!

Example of Calculating running time Sum of n numbers • Sum(n) Init Sum ← 0 ------------ 1 time For i ← 1 To n -------------- n+1 times Sum ← Sum+i -------- n times Return Sum --------------- 1 time • 1+n+1=2 n+3 = ignoring constant, it is n = linear time • If n is 5 then time is 13

Assignment # 1 • Write an algorithm to call your friend. • Write an algorithm for searching ‘Algorithm’s text book’ from Library? • How does one get to see the film “THREE IDIOTS” in cinema?