Design Analysis of Algorithms Introduction Introduction Algorithms are

Design & Analysis of Algorithms Introduction

Introduction • Algorithms are the ideas behind computer programs. • An algorithm is the thing which stays the same whether the program is in C++ running on a PC in New York or is in Android running on a Smartphone in Lahore!

What is Algorithm? • Sequence of steps that can be taken to solve a problem • An algorithm is a sequence of unambiguous instructions for solving a problem in a finite amount of time. • An Algorithm is well defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values as output. • More generally, an Algorithm is any well defined computational procedure that takes collection of elements as input and produces a collection of elements as output. Input Algorithm output

Properties of Algorithms • It must be correct. • It must be composed of a series of concrete steps. • The execution sequence of instructions should not be ambiguous. • It must have finite number of instructions and steps. • It must terminate. 4

Syntax & Semantics An algo. is “correct” if its: WARNINGS: – Semantics are correct 1. An algo. can be – Syntax is correct syntactically correct, yet semantically incorrect – very Semantics: The concept embedded in dangerous situation! an algorithm (the soul!) 2. Syntactic correctness is easier Syntax: to check as The actual representation compared with of an algorithm (the body!) semantic

Solving Problems (1) When faced with a problem: 1. We first clearly define the problem 2. Think of possible solutions 3. Select the one that we think is the best under the prevailing circumstances 4. And then apply that solution 5. If the solution woks as desired, fine; else we go back to step 2

Early History: Search for a Generic Algorithm • The study of algorithms began with mathematicians and was a significant area of work in the early years • The goal of those early studies was to find a single, general algorithm that could solve all problems of a single type

Origin of the Term “Algorithm” • The name derives from the title of a Latin book: Algoritmi de numero Indorum • That book was a translation of an Arabic book: Al-Khwarizmi Concerning the Hindu Art of Reckoning • That book was written by the famous 9 -th century Muslim mathematician, Muhammad ibn Musa al-Khwarizmi

Al-Khwarzmi • Al-Khwarizmi lived in Baghdad, where he worked at the Dar al-Hikma • Dar al-Hikma acquired and translated books on science and philosophy, particularly those in Greek, as well as publishing original research • The word Algebra has its origins in the title of another Latin book which was a translation of yet another book written by Al-Khwarzmi: Kitab al-Mukhtasar fi Hisab al-Jabr wa'l-Muqabala

Al-Khwarizmi’s Golden Principle All complex problems can be and must be solved using the following simple steps: 1. Break down the problem into small, simple sub -problems 2. Arrange the sub-problems in such an order that each of them can be solved without effecting any other 3. Solve them separately, in the correct order 4. Combine the solutions of the sub-problems to form the solution of the original problem

Algorithms to Programs Problem Algorithm: A sequence of instructions describing how to do a task 11

Difference b/w Algorithm and a program • Algorithm • It is a method followed to solve a problem. • A mapping of input to output. As mentioned on slide no. 2. • A problem can have many algorithms • Computer program • It is a concrete representation of an algorithm in some programming language • A set of instructions which the computer will follow to solve a problem 12

Popular Algorithms, Factors of Dependence • Most basic and popular algorithms are – Sorting algorithms – Searching algorithms Which algorithm is best? • Mainly, it depends upon various factors, for example in case of sorting – The number of items to be sorted – The extent to which the items are already sorted – Possible restrictions on the item values – The kind of storage device to be used etc.

One Problem, Many Algorithms Problem • The statement of the problem specifies, in general terms, the desired input/output relationship. Algorithm • The algorithm describes a specific computational procedure for achieving input/output relationship. Example • One might need to sort a sequence of numbers into non-decreasing order. Algorithms • Various algorithms e. g. merge sort, quick sort, heap sorts etc.

Important Designing Techniques • Brute Force – Straightforward, naive approach – Mostly expensive • Divide-and-Conquer – Divide into smaller sub-problems • Iterative Improvement – Improve one change at a time • Decrease-and-Conquer – Decrease instance size • Transform-and-Conquer – Modify problem first and then solve it

Important Designing Techniques • Greedy Approach – – – Locally optimal decisions, can not change once made. Efficient Easy to implement The solution is expected to be optimal Every problem may not have greedy solution • Dynamic programming – Decompose into sub-problems like divide and conquer – Sub-problems are dependant – Record results of smaller sub-problems – Re-use it for further occurrence – Mostly reduces complexity exponential to polynomial There is a Tradeoffs b/w Space and Time – Use more space now to save time later

Problem Solving Process • Problem • Strategy • Algorithm – Input – Output – Steps • Analysis – Correctness – Time & Space – Optimality • Implementation • Verification