IAT 800 Recursion Web Crawling Fall 2006 IAT

IAT 800 Recursion, Web Crawling , Fall 2006 IAT 800

Today’s Nonsense g Recursion – Why is my head spinning? g Web Crawling – Recursing in HTML , Fall 2006 IAT 800 Shortened class again today. Boy, your TA must be a total slacker, or something.

Recursion g Recursion basically means calling a method from inside itself. What the? !? int factorial(int n) { if(n > 1) { return n * factorial(n-1); } else return 1; } , Fall 2006 IAT 800

Inside Itself? ! g Let’s step through what happens. factorial(3); , Fall 2006 int factorial(int n) { (n = 3) if(n > 1) { return n * factorial(n-1); } else return 1; } IAT 800

Inside Itself? ! g Let’s factorial(3); step through what happens. int factorial(int n) { (n = 3) if(n > 1) { return n * factorial(n-1); } else return 1; } int factorial(int n) { (n = 2) if(n > 1) { return n * factorial(n-1); } else return 1; } , Fall 2006 IAT 800

Inside Itself? ! g Let’s factorial(3); step through what happens. int factorial(int n) { (n = 3) if(n > 1) { return n * factorial(n-1); } else return 1; } int factorial(int n) { (n = 2) if(n > 1) { return n * factorial(n-1); } else return 1; } int factorial(int n) { (n = 1) if(n > 1) { return n * factorial(n-1); } else return 1; } , Fall 2006 IAT 800

Inside Itself? ! g Let’s factorial(3); step through what happens. int factorial(int n) { (n = 3) if(n > 1) { return n * factorial(n-1); } else return 1; } int factorial(int n) { (n = 2) if(n > 1) { return n * factorial(n-1); } else return 1; } 1 , Fall 2006 IAT 800

Inside Itself? ! g Let’s factorial(3); step through what happens. int factorial(int n) { (n = 3) if(n > 1) { return n * factorial(n-1); } else return 1; } int factorial(int n) { if(n > 1) { return n * 1; } else return 1; } , Fall 2006 IAT 800 (n = 2)

Inside Itself? ! g Let’s factorial(3); step through what happens. int factorial(int n) { (n = 3) if(n > 1) { return n * factorial(n-1); } else return 1; } int factorial(int n) { if(n > 1) { return 2 * 1; } else return 1; } , Fall 2006 IAT 800 (n = 2)

Inside Itself? ! g Let’s factorial(3); step through what happens. int factorial(int n) { (n = 3) if(n > 1) { return n * factorial(n-1); } else return 1; } 2 , Fall 2006 IAT 800

Inside Itself? ! g Let’s step through what happens. factorial(3); , Fall 2006 int factorial(int n) { if(n > 1) { return n * 2; } else return 1; } (n = 3) IAT 800

Inside Itself? ! g Let’s step through what happens. factorial(3); , Fall 2006 int factorial(int n) { if(n > 1) { return 3 * 2; } else return 1; } (n = 3) IAT 800

Inside Itself? ! g Let’s step through what happens. factorial(3); , Fall 2006 6 IAT 800

Base Case The most important thing to include in a recursive call is the “base case”, something that will assure that the function stops calling itself at some point. g In our example, we made sure it only called the recursive function if n was greater than 1, and each time we call it, n gets smaller, so we know it will eventually get to a number less than or equal to 1. g , Fall 2006 IAT 800

Web Crawling Let’s use recursion for something more interesting. g Say we have some method “parse. Page”, that looks at a web page. Suppose we then want that method to follow the links on that page and parse the pages it is linked to. g We’d then want to call the “parse. Page” method on those links from inside the parse. Page method we have. g , Fall 2006 IAT 800

Web Crawling , Fall 2006 IAT 800

Web Crawling g You can see here the need for a base case. One way of controlling our search is by placing a limit on the depth of the links we follow. g For instance, in the visual example, we followed the links from our start page (depth 1), and then the links from those pages (depth 2). , Fall 2006 IAT 800

Recursion Fig. 3: Remember— base cases prevent infinite cats. http: //infinitecat. com/ , Fall 2006 IAT 800

Parse HTML? g “Parsing” means to walk through the structure of a file (look at it word-by-word) – Look at an HTML file g Results 1 - 20 of about 202 for <a href=/url? sa=X&oi=dict&q=http: //www. answers. com/matrix%26 r%3 D 67 title="Look up definition of matrix">matrix</a> <a href=/url? sa=X&oi=dict&q=http: //www. answers. com/red%26 r%3 D 67 title="Look up definition of red">red</a> … The structure of an HTML file is the tag structure – So parsing means to walk through and interpret the tags some text == some text g If you can parse HTML files, you can pull content out of web pages and do stuff with it g Procedural manipulation of web content , Fall 2006 IAT 800

Basic approach g Use two classes to parse g One class reads info from a URL – HTMLParser The other class is used by HTMLParser to process tags – child of HTMLEditor. Kit. Parser. Callback g g HTMLParser recognizes when a tag appears (<TAG>) and calls appropriate methods on the Parser. Callback class (start-tags, end -tags, simple-tags, text, etc. ) g The programmer (ie. you), fill in the Parser. Callback methods to do whatever you want when you see different kinds of tags , Fall 2006 IAT 800

Running the example g We’ve written HTMLParser for you g To access it, it must be in the data directory of your project g Simplest thing will be just to copy the code from the website and put the directory in your default sketchbook directory , Fall 2006 IAT 800

handle. Simple. Tag g public void handle. Simple. Tag(HTML. Tag tag, Mutable. Attribute. Set attrib, int pos) – – g Called for tags like IMG tag stores the name of the tag attrib stores any attributes pos is the position in the file Example: <img src=“image. gif” alt=“text description of image” align=“right” width=“ 10”> – The tag is img – The attributes are src, alt, align, width (with their respective values) , Fall 2006 IAT 800

handle. Start. Tag g public void handle. Start. Tag(HTML. Tag tag, Mutable. Attribute. Set attrib, int pos) – – g Called for tags like BODY tag stores the name of the tag attrib stores any attributes pos is the position in the file Example: <body bgcolor=“#FFFFFF” topmargin=“ 0” leftmargin=“ 0” marginheight=“ 0” marginwidth=“ 0”> – The tag is body – The attributes are bgcolor, topmargin, leftmargin, marginheight (with their respective values) , Fall 2006 IAT 800

handle. End. Tag g public void handle. End. Tag(HTML. Tag tag, int pos) – Called for tags like </a> – tag stores the name of the tag – pos is the position in the file , Fall 2006 IAT 800

handle. Text g public void handle. Text(char[] data, int pos) – Handles anything that’s not a tag (the text between tags) – data is an array of characters containing the text – pos is the position , Fall 2006 IAT 800

Filling in these methods g You fill in these methods to do whatever processing you want g In the image collage example – handle. Simple. Tag is looking for images – handle. Start. Tag is looking for the start of anchors and follows links , Fall 2006 IAT 800