Python Tutorial Adapted from slides by Prof Mitch

Python Tutorial Adapted from slides by Prof. Mitch Marcus

Python Python is an open source scripting language. Developed by Guido van Rossum in the early 1990 s Named after Monty Python Available for download from http: //www. python. org

Why Python? Object Oriented With functional constructs: map, generators, list comprehension NLP Processing: Symbolic Python has built-in datatypes for strings, lists, and more NLP Processing: Statistical Python has strong numeric processing capabilities: matrix operations, etc. Suitable for probability and machine learning code. NLTK: Natural Language Tool Kit Widely used for teaching NLP First developed for this course Implemented as a set of Python modules Provides adequate libraries for many NLP building blocks Google "NLTK" for more info, code, data sets, book. .

Why Python? Powerful but unobtrusive object system Every value is an object Classes guide but do not dominate object construction Powerful collection and iteration abstractions Dynamic typing makes generics easy

Python Interpreted language: works with an evaluator for language expressions Dynamically typed: variables do not have a predefined type Rich, built-in collection types: Lists Tuples Dictionaries (maps) Sets Concise

Language features Indentation instead of braces Newline separates statements Several sequence types Strings '…': made of characters, immutable Lists […]: made of anything, mutable Tuples (…) : made of anything, immutable Powerful subscripting (slicing) Functions are independent entities (not all functions are methods) Exceptions

Dynamic typing Java/C++: statically typed Variables are declared to refer to objects of a given type Methods use type signatures to enforce contracts Python Variables come into existence when first assigned to A variable can refer to an object of any type All types are (almost) treated the same way Main drawback: type errors are only caught at runtime

Playing with Python (1) >>> 2+3 5 >>> 2/3 0 >>> 2. 0/3 0. 666666663 >>> x=4. 5 >>> int(x) 4

Playing with Python (2) >>> x='abc' >>> x[0] 'a' >>> x[1: 3] 'bc' >>> x[: 2] 'ab' >>> x[1]='d' Traceback (most recent call last): File "<pyshell#14>", line 1, in <module> x[1]='d' Type. Error: 'str' object does not support item assignment

Playing with Python (4) >>> def p(x): if len(x) < 2: return True else: return x[0] == x[-1] and p(x[1: -1]) >>> p('abc') False >>> p('aba') True >>> p([1, 2, 3]) False >>> p([1, 'a', 1]) True >>> p((False, 2, 2, False)) True >>> p(('a', 1, 1)) False

Python dictionaries (Maps) >>> d={'alice': 1234, 'bob': 5678, 'clare': 9012} >>> d['alice'] 1234 >>> d['bob'] 5678 >>> d['bob'] = 7777 >>> d {'clare': 9012, 'bob': 7777, 'alice': 1234} >>> d. keys() ['clare', 'bob', 'alice'] >>> d. items() [('clare', 9012), ('bob', 7777), ('alice', 1234)] >>> del d['bob'] >>> d {'clare': 9012, 'alice': 1234}

Installing & Running Python

The Python Interpreter Interactive interface to Python % python Python 2. 6 (r 26: 66714, Feb 3 2009, 20: 49) [GCC 4. 3. 2 [gcc-4_3 -branch revision 141291]] on linux 2 Type "help", "copyright", "credits" or "license" for more information. >>> Python interpreter evaluates inputs: >>> 3*(7+2) 27

IDLE Development Environment Shell for interactive evaluation. Text editor with color-coding and smart indenting for creating Python files. Menu commands for changing system settings and running files.

Running Interactively on Linux On Unix… % python >>> 3+3 6 Python prompts with '>>>'. To exit Python (not Idle): In Unix, type CONTROL-D In Windows, type CONTROL-Z + <Enter>

Running Programs on UNIX % python filename. py You can make a python file executable by adding following text as the first line of the file to make it runnable: #!/usr/bin/python

The Basics

A Code Sample x = 34 - 23 # A comment. y = "Hello" # Another one. z = 3. 45 if z == 3. 45 or y == "Hello": x = x + 1 y = y + " World" # String concat. print x print y

Enough to Understand the Code Indentation matters to the meaning of the code: Block structure indicated by indentation The first assignment to a variable creates it. Variable types don't need to be declared. Python figures out the variable types on its own. Assignment uses = and comparison uses ==. For numbers + - * / % are as expected. Special use of + for string concatenation. Special use of % for string formatting (as with printf in C) Logical operators are words (and, or, not) not symbols Simple printing can be done with print.

Basic Datatypes Integers (default for numbers) z = 5 / 2 # Answer is 2, integer division. Floats x = 3. 456 Strings Can use "" or '' to specify. "abc" 'abc' (Same thing. ) Unmatched can occur within the string. "matt's" Use triple double-quotes for multi-line strings or strings than contain both ' and " inside of them: """a'b"c"""

Whitespace is meaningful in Python: especially indentation and placement of newlines. Use a newline to end a line of code. Use when must go to next line prematurely. No braces { to mark blocks of code in Python… Use consistent indentation instead. } The first line with less indentation is outside of the block. The first line with more indentation starts a nested block Often a colon appears at the start of a new block. (E. g. for function and class definitions. )

Comments Start comments with # – the rest of line is ignored. Can include a "documentation string" as the first line of any new function or class that you define. The development environment, debugger, and other tools use it: it's good style to include one def my_function(x, y): """This is the docstring. This function does blah. """ # The code would go here. . .

Assignment Binding a variable in Python means setting a name to hold a reference to some object. Assignment creates references, not copies Variables in Python do not have an intrinsic type. Objects have types. Python determines the type of the reference automatically based on what data is assigned to it. You create a variable the first time it appears on the left side of an assignment expression: x = 3 A reference is deleted via garbage collection after any variables bound to it have passed out of scope Python uses reference semantics

Reference Semantics Variable contain a pointer to the location in memory that stores the value l Example: l l 1 = [1, 2, 3] l 2 = l 1[0] = 0 l 2[0] == 0 ?

Value Semantics Variables contain the value, assignment copies the value l Example: l x=5 y=x x=7 x == y ? l Note: assignment is involved also when passing parameters

Naming Rules Names are case sensitive and cannot start with a number. They can contain letters, numbers, and underscores. bob Bob _bob _2_bob_2 Bo. B There are some reserved words: and, assert, break, class, continue, def, del, elif, else, except, exec, finally, for, from, global, if, import, in, is, lambda, not, or, pass, print, raise, return, try, while

Accessing Non-Existent Name Trying to access a name before it's been properly created (by placing it on the left side of an assignment), gives an error: >>> y Traceback (most recent call last): File "<pyshell#16>", line 1, in -toplevely Name. Error: name 'y' is not defined >>> y = 3 >>> y 3

Sequence types Tuples, Lists and Strings

Sequence Types 1. Tuple A simple immutable ordered sequence of items Items can be of mixed types, including collection types 2. Strings Immutable Conceptually very much like a tuple (8 -bit characters. Unicode strings use 2 -byte characters. ) 3. List Mutable ordered sequence of items of mixed types

Similar Syntax All three sequence types (tuples, strings, and lists) share much of the same syntax and functionality. Key difference: Tuples and strings are immutable Lists are mutable The operations shown in this section can be applied to all sequence types most examples will just show the operation performed on one

Sequence Types Tuples are defined using parentheses (and commas). >>> tu = (23, 'abc', 4. 56, (2, 3), 'def') Lists are defined using square brackets (and commas). >>> li = ["abc", 34, 4. 34, 23] Strings are defined using quotes (", ', or """). >>> st string = "Hello World" = 'Hello World' = """This is a multi-line that uses triple quotes. """

Sequence Types We can access individual members of a tuple, list, or string using square bracket "array" notation. Note that all are 0 based… >>> tu = (23, 'abc', 4. 56, (2, 3), 'def') >>> tu[1] # Second item in the tuple. 'abc' >>> li = ["abc", 34, 4. 34, 23] >>> li[1] # Second item in the list. 34 >>> st = "Hello World" >>> st[1] # Second character in string. 'e'

Positive and negative indices >>> t = (23, 'abc', 4. 56, (2, 3), 'def') Positive index: count from the left, starting with 0. >>> t[1] 'abc' Negative lookup: count from right, starting with – 1. >>> t[-3] 4. 56

Slicing: Return Copy of a Subset >>> t = (23, 'abc', 4. 56, (2, 3), 'def') Return a copy of the container with a subset of the original members. Start copying at the first index, and stop copying before the second index. >>> t[1: 4] ('abc', 4. 56, (2, 3)) You can also use negative indices when slicing. >>> t[1: -1] ('abc', 4. 56, (2, 3))

Slicing: Return Copy of a Subset >>> t = (23, 'abc', 4. 56, (2, 3), 'def') Omit the first index to make a copy starting from the beginning of the container. >>> t[: 2] (23, 'abc') Omit the second index to make a copy starting at the first index and going to the end of the container. >>> t[2: ] (4. 56, (2, 3), 'def')

The 'in' Operator Boolean test whether a value is inside a collection (often called a container in Python: >>> t >>> 3 False >>> 4 True >>> 4 False = [1, 2, 4, 5] in t not in t For strings, tests for substrings >>> a = 'abcde' >>> 'c' in a True >>> 'cd' in a True >>> 'ac' in a False Be careful: the in keyword is also used in the syntax of for loops and list comprehensions.

The + Operator The + operator produces a new tuple, list, or string whose value is the concatenation of its arguments. >>> (1, 2, 3) + (4, 5, 6) (1, 2, 3, 4, 5, 6) >>> [1, 2, 3] + [4, 5, 6] [1, 2, 3, 4, 5, 6] >>> "Hello" + "World" 'Hello World'

Mutability: Tuples vs. Lists

Lists: Mutable >>> li = ['abc', 23, 4. 34, 23] >>> li[1] = 45 >>> li ['abc', 45, 4. 34, 23] We can change lists in place. Name li still points to the same memory reference when we're done.

Tuples: Immutable >>> t = (23, 'abc', 4. 56, (2, 3), 'def') >>> t[2] = 3. 14 Traceback (most recent call last): File "<pyshell#75>", line 1, in -topleveltu[2] = 3. 14 Type. Error: object doesn't support item assignment You can't change a tuple. You can make a fresh tuple and assign its reference to a previously used name. >>> t = (23, 'abc', 3. 14, (2, 3), 'def') The immutability of tuples means they're faster than lists.

Operations on Lists Only (1) >>> li = [1, 11, 3, 4, 5] >>> li. append('a') # Note the method syntax >>> li [1, 11, 3, 4, 5, 'a'] >>> li. insert(2, 'i') >>>li [1, 11, 'i', 3, 4, 5, 'a']

The extend method vs the + operator + creates a fresh list (with a new memory reference) extend operates on list li in place. >>> li. extend([9, 8, 7]) >>> li [1, 2, 'i', 3, 4, 5, 'a', 9, 8, 7] Confusing: extend takes a list as an argument. append takes a singleton as an argument. >>> li. append([10, 11, 12]) >>> li [1, 2, 'i', 3, 4, 5, 'a', 9, 8, 7, [10, 11, 12]]

Operations on Lists Only (3) >>> li = ['a', 'b', 'c', 'b'] >>> li. index('b') 1 *more # index of first occurrence* complex forms exist >>> li. count('b') 2 # number of occurrences >>> li. remove('b') >>> li ['a', 'c', 'b'] # remove first occurrence

Operations on Lists Only (4) >>> li = [5, 2, 6, 8] >>> li. reverse() >>> li [8, 6, 2, 5] # reverse the list *in place* >>> li. sort() >>> li [2, 5, 6, 8] # sort the list *in place* >>> li. sort(some_function) # sort in place using user-defined comparison

Summary: Tuples vs. Lists slower but more powerful than tuples. Lists can be modified, and they have lots of handy operations we can perform on them. Tuples are immutable and have fewer features. To convert between tuples and lists use the list() and tuple() functions: li = list(tu) tu = tuple(li)

Dictionaries: a mapping collection type

Dictionaries: A Mapping type Dictionaries store a mapping between a set of keys and a set of values. Keys can be any immutable type. Values can be any type Values and keys can be of different types in a single dictionary You can define modify view lookup delete the key-value pairs in the dictionary.

Creating and accessing dictionaries >>> d = {'user': 'bozo', 'pswd': 1234} >>> d['user'] 'bozo' >>> d['pswd'] 1234 >>> d['bozo'] Traceback (innermost last): File '<interactive input>' line 1, in ? Key. Error: bozo

Updating Dictionaries >>> d = {'user': 'bozo', 'pswd': 1234} >>> d['user'] = 'clown' >>> d {'user': 'clown', 'pswd': 1234} Keys must be unique. Assigning to an existing key replaces its value. >>> d['id'] = 45 >>> d {'user': 'clown', 'id': 45, 'pswd': 1234} Dictionaries are unordered New entry might appear anywhere in the output. (Dictionaries work by hashing)

Removing dictionary entries >>> d = {'user': 'bozo', 'p': 1234, 'i': 34} >>> del d['user'] >>> d {'p': 1234, 'i': 34} # Remove one. >>> d. clear() >>> d {} # Remove all. >>> a = [1, 2] >>> del a[1] >>> a [1] # (del also works on lists)

Useful Accessor Methods >>> d = {'user': 'bozo', 'p': 1234, 'i': 34} >>> d. keys() ['user', 'p', 'i'] # List of current keys >>> d. values() ['bozo', 1234, 34] # List of current values >>> d. items() # List of item tuples. [('user', 'bozo'), ('p', 1234), ('i', 34)]

Functions in Python (Methods later)

Python and Types Python determines the types of variable bindings in a program automatically. "Dynamic Typing" But objects have types and Python enforces the types of objects. "Strong Typing" So, for example, you can't just append an integer to a string. You must first convert the integer to a string itself. x = "the answer is " y = 23 print x + y # Decides x is bound to a string. # Decides y is bound to an integer. # Python will complain about this.

Calling a Function The syntax for a function call is: >>> def myfun(x, y): return x * y >>> myfun(3, 4) 12 Parameters are passed by value (“Call by Value”) Each argument expression is evaluated and the resulting value is bound to the corresponding variable in the function (which is a new one for each invocation) All assignment in Python, including binding function parameters, use reference semantics, since values are all object references. Many web discussions on this are simply confused and call it “call by assignment” or “call by sharing”.

Functions without returns All functions in Python have a return value even if no return line inside the code. Functions without a return the special value None is a special constant in the language. None is used like NULL, void, or nil in other languages. None is also logically equivalent to False. The interpreter doesn't print None

Function overloading? No. There is no function overloading in Python. Unlike C++, a Python function is specified by its name alone The number, order, names, or types of its arguments cannot be used to distinguish between two functions with the same name. Two different functions can't have the same name, even if they have different arguments. But: see operator overloading in later slides

Functions are first-class objects in Python Functions can be used as any other data type They can be Arguments to function Return values of functions Assigned to variables Parts of tuples, lists, etc … >>> def myfun(x): return x*3 >>> def apply(q, x): return q(x) >>> apply(myfun, 7) 21

Logical Expressions

True and False are constants in Python. Other values equivalent to True and False: zero, None, empty container or object True: non-zero numbers, non-empty objects Comparison operators: ==, !=, <, <=, etc. X and Y have same value: X == Y Compare with X is Y : X and Y are two variables that refer to the identical same object.

Boolean Logic Expressions You can also combine Boolean expressions. True if a is True and b is True: True if a is True or b is True: True if a is False: a and b a or b not a Use parentheses as needed to disambiguate complex Boolean expressions. Actually, evaluation of expressions is lazy…

Special Properties of and or Actually and or don't return True or False. They return the value of one of their sub-expressions (which may be a non-Boolean value). X and Y and Z If all are true, returns value of Z. Otherwise, returns value of first false sub-expression. X or Y or Z If all are false, returns value of Z. Otherwise, returns value of first true sub-expression. and or use lazy evaluation, so no further expressions are evaluated

Conditional Expressions x = true_value if condition else false_value Evaluation rule: First, condition is evaluated If True, true_value is evaluated and returned If False, false_value is evaluated and returned

Control of Flow

if Statements if x == 3: print "X equals elif x == 2: print "X equals else: print "X equals print "This is outside 3. " 2. " something else. " the 'if'. " Be careful! The keyword if is also used in the syntax of filtered list comprehensions. Note: Use of indentation for blocks Colon (: ) after boolean expression

while Loops >>> x = 3 >>> while x < 5: print x, "still in the loop" x = x + 1 3 still in the loop 4 still in the loop >>> x = 6 >>> while x < 5: print x, "still in the loop" >>>

break and continue You can use the keyword break inside a loop to leave the while loop entirely. You can use the keyword continue inside a loop to stop processing the current iteration of the loop and to immediately go on to the next one.

assert An assert statement will check to make sure that something is true during the course of a program. If the condition if false, the program stops (more accurately: the program throws an exception) assert(number_of_players < 5)

For Loops

For Loops / List Comprehensions Python's list comprehensions provide a natural idiom that usually requires a for-loop in other programming languages. As a result, Python code uses many fewer for-loops Nevertheless, it's important to learn about for-loops. Caveat! The keywords for and in are also used in the syntax of list comprehensions, but this is a totally different construction.

For Loops 1 For-each is Python's only for construction A for loop steps through each of the items in a collection type, or any other type of object which is "iterable" for <item> in <collection>: <statements> If <collection> is a list or a tuple, then the loop steps through each element of the sequence. If <collection> is a string, then the loop steps through each character of the string. for some. Char in "Hello World": print some. Char

For Loops 2 for <item> in <collection>: <statements> <item> can be more complex than a single variable name. When the elements of <collection> are themselves sequences, then <item> can match the structure of the elements. This multiple assignment can make it easier to access the individual parts of each element. for (x, y) in [(a, 1), (b, 2), (c, 3), (d, 4)]: print x

For loops and the range() function Since a variable often ranges over some sequence of numbers, the range() function returns a list of numbers from 0 up to but not including the number we pass to it. range(5) returns [0, 1, 2, 3, 4] So we could say: for x in range(5): print x (There are more complex forms of range() that provide richer functionality…) xrange() returns an iterator that provides the same functionality here more efficiently

For Loops and Dictionaries >>> ages = { "Sam" : 4, "Mary": 3, "Bill": 2 } >>> ages {'Bill': 2, 'Mary': 3, 'Sam': 4} >>> for name in ages. keys(): print name, ages[name] Bill 2 Mary 3 Sam 4 >>> for (name, age) in ages. items(): print name, age

String Operations A number of methods for the string class perform useful formatting operations: >>> "hello". upper() 'HELLO' Check the Python documentation for many other handy string operations. Helpful hint: use <string>. strip() to strip off final newlines from lines read from files

Printing with Python You can print a string to the screen using "print" Using the % string operator in combination with the print command, we can format our output text. >>> print "%s xyz %d" abc xyz 34 % ("abc", 34) "Print" automatically adds a newline to the end of the string. If you include a list of strings, it will concatenate them with a space between them. >>> print "abc" abc Useful trick: >>> single space >>> print "abc", "def" abc def print "abc", doesn't add newline just a

Convert Anything to a String The built-in str() function can convert an instance of any data type into a string. You can define how this function behaves for user-created data types. You can also redefine the behavior of this function for many types. >>> "Hello " + str(2) "Hello 2"

Importing and Modules

Importing and Modules Use classes & functions defined in another file. A Python module is a single file with the same name (plus the. py extension) Like Java import Where does Python look for module files? The list of directories where Python looks: sys. path To add a directory of your own to this list, append it to this list. sys. path. append('/my/new/path')

Import (1) import somefile Everything in somefile. py can be referred to by: somefile. class. Name. method("abc") somefile. my. Function(34) from somefile import * Everything in somefile. py can be referred to by: class. Name. method("abc") my. Function(34) Caveat! This can easily overwrite the definition of an existing function or variable!

Import (2) from somefile import class. Name Only the item class. Name in somefile. py gets imported. Refer to it without a module prefix. Caveat! This can overwrite an existing definition. class. Name. method("abc") This was imported my. Function(34) Not this one

Commonly Used Modules Some useful modules to import, included with Python: Module: sys - Lots of handy stuff. Maxint Module: os. path - OS specific code. - Directory processing.

More Commonly Used Modules Module: math - Mathematical code. Exponents sqrt Module: Random - Random number code. Randrange Uniform Choice Shuffle To see what's in the standard library of modules, check out the Python Library Reference: http: //docs. python. org/lib. html Or O'Reilly's Python in a Nutshell: http: //proquest. safaribooksonline. com/0596100469 (URL works inside of UPenn, afaik, otherwise see the course web page)

Object Oriented Programming in Python: Defining Classes

It's all objects… Everything in Python is really an object. We've seen hints of this already… "hello". upper() list 3. append('a') dict 2. keys() These look like Java or C++ method calls. Programming in Python is typically done in an object oriented fashion.

Defining a Class A class is a special data type which defines how to build a certain kind of object. The class also stores some data items that are shared by all the instances of this class. But no static variables! Python doesn't use separate class interface definitions much. You just define the class and then use it.

Methods in Classes Define a method in a class by including function definitions within the scope of the class block. There must be a special first argument self in all of method definitions which gets bound to the calling instance. self is like this in Java self always refers to the current class instance A constructor for a class is a method called __init__ defined within the class.

A simple class definition: Person class Person(Object): """A class representing a person. """ population = 0 def __init__(self, n, a): self. full_name = n self. age = a Person. population += 1 def __del__(self): Person. population -=1 def get_age(self): return self. age

Inheritance: Student class Student(Person): """A class representing a student. """ def __init__(self, n, a, id): super(Person, self). __init__(n, a) self. id = id def get_id(self): return self. id

Creating and Deleting Instances

Instantiating Objects There is no "new" keyword as in Java. Merely use the class name with () notation and assign the result to a variable. __init__ serves as a constructor for the class. Example: b = student("Bob", 21) An __init__ method can take any number of arguments. Like other functions & methods, arguments can be defined with default values, making them optional to the caller. But no real overloading

Self Although you must specify self explicitly when defining the method, you don't include it when calling the method. Python passes it for you automatically. Defining a method: Calling a method: (this code inside a class definition. ) def set_age(self, num): self. age = num >>> x. set_age(23)

Access to Attributes and Methods

Traditional Syntax for Access >>> f = Person("Bob Smith", 23) >>> f. full_name "Bob Smith" # Access an attribute. >>> f. get_age() 23 # Access a method. No public, private, protected, etc…

File Processing, Error Handling, Regular Expressions, etc.

File Processing with Python This is a good way to play with the error handing capabilities of Python. Try accessing files without permissions or with non-existent names, etc. You'll get plenty of errors to look at and play with! file = open('filename') somestring = file. read() for line in file: print line file. close() For more, see section 3. 9 of the Python Library reference at http: //docs. python. org/lib/bltin-file-objects. html

Exception Handling Exceptions are Python classes More specific kinds of errors are subclasses of the general Error class. You use the following commands to interact with them: Try Except Finally Catch

Regular Expressions and Match Objects Python provides a very rich set of tools for pattern matching against strings in module re (for regular expression) For a gentle introduction to regular expressions in Python see http: //www. diveintopython. org/regular_expressions/index. html or http: //www. amk. ca/python/howto/regex. html

Regular Expressions import re quote_re = re. compile('". +"') quote_re. search('then he said "find the quote". . . ') X = quote_re. search('then he said "find the quote". . . ') X. group() '"find the quote"' X = quote_re. search('quote 1: "im a quote" quote 2: "so am i" more words') X. group() '"im a quote" quote 2: "so am i"' quote_re 2 = re. compile('"[^"]+"') X = quote_re 2. search('quote 1: "im a quote" quote 2: "so am i" more words') X. group() '"im a quote"' sent = 'quote 1: "im a quote" quote 2: "so am i" more words' quote_re 2. findall(sent) ['"im a quote"', '"so am i"'] quote_re 3 = re. compile('". +? "') X = quote_re 3. search(sent) X. group() '"im a quote"'