Lecture 12 Odds and Ends for Project 5

Lecture 12 • Odds and Ends for Project 5 • Final Presentation Schedule & Grades – PS: grades for past projects soon • Odds and Ends for Tech • Next time – Mano Talk, much Geo Goodness

Project 5 • Up your game – More than a mashup, this is IO/IR. (sparta? ) – Stats are good to start, finding signal in noise is good, infovis is good. Conclusions rock. New ways of doing IO/IR rock. – If someone could re-create your project in a few minutes using Yahoo Pipes, worry. (but cool to use pipes!) – Good not to do the old option #1 of building off the 202 project – considering they killed that project. But TF/IDF is still a good option. SOLR! my. SQL built in search! Etc etc. – Doesn't have to be a website (smart browser plugin, mobile native app, paper, presentation, etc) – AS LONG AS HEAVY INFORMATION FOCUS AND CLEARED WITH ME • Everything in SVN – Creative, code, notes, everything. – Not just at the end. This is how we spy on your progress. – This is also how you CYA if everything blows up the last week. "See Mr. Hill, we didn't leave it all to the end!"

Teams and Goals Who What Knowns Unknowns Donghyuk Rami Anne Alice identify trends within communities using hashtags Worked with twitter b 4 ? Team Matt Gilbert Deepak Brendan Andrew Secret Santa Web + Mobile connect to a "social network" site scanning of the user profile NLP and shopping api's match up gift recommendations Made Android ? app Gaurav Emily Vimal Iris New (Facetted? Tabular? ) search for Wikipedia dumps. Zemanta Is new? Will it be good? Andrea Ariel Gregory Jen ? ? ? Natarajan Elliot Karthik sam mobile voting app for TT auth for "one-person, one vote"

Dates • Monday, Dec 12, 5: 30 – Project 5 due (yes, before you demo it. Knives Down. ) – One more class (TBD Content – Need Input!) • Showcase on Thursday, Dec 15 2011, 3: 30 -4: 30 – 4: 30 -5: Kick guests out, and teams that didn't demo #5 get to demo to Benjamin and Karen • Grades – Baseline + Projects + Participation (class) + Participation (online) = final grade – Everyone is doing at least "OK" (Kinda how Grad School tends to work)

Grade IO Research • For Project X – Did Team Member Y (not yourself) • Contribute (less|avg|more) EFFORT than team avg to final product, after taking into consideration the person's starting point – Then you get what everyone (on average) thought of your help • Is it an – Interesting way for you to get self-evaluation – or horrible idea that will cause real-housewives-ofsouth-hall fights? • What if you were in a 2 -person team?

Showcase • We are inviting the world – Rest of i. School – Friends – IO-Lab supporters • … But no one will come if the invites don't come from you! • A week ahead, you need to choose: What project do you want to present? – Then we see if everything fits, because that could mean you are dragging in old teamates… • Upload ALL projects to SVN, from 1 to 5

Tech Odds and Ends • Where we started: Memex Trails in Java. Script and Delicious • And then we added on – Java. Script, j. Query, Browser Plugins, JSON, JS OOP – HTML 5, DOM Events, Adding Elements, Local. Storage, Geo. Location – CSS, CSS layout, CSS 3, Box Model, Info. Viz tools, j. Query. UI – Callbacks, Anonymous Functions, AJAX, JSON-P, Proxies – Mobile, Phonegap, Mobile Web Apps, Mobile UX – SVN, GIT – Infoviz, Raster vs Vector – Map/Reduce, 80 legs, Mechanical Turk, solar – App. Engine, Data Storage • Where we (hopefully) ended: Knowing right level of difficulty of IO/IR tech, and starting points for digging deeper

What we Glossed Over • • • Other browser extensions (like Grease. Monkey) RDF and FOAF, Freebase, Microformats <link rel="canonical" Regular Expressions Deeper dive into Info. Vis Libraries Scraping, Beautiful. Soup SMS, zeepmobile. com, twilio. com No. SQL, memcache Geo. Spacial (Next Time!) Test Driven Development, Unit Tests

Regular Expressions • • • Easy way to describe a pattern /a/ -- look for an "a" /. a/ -- any character followed by an a /abc? d/ -- abcd or abd /^abc$/ -- special chars "starts with" and "ends with" (glues to edges) /[^a]b/ -- anything other than an "a" then a "b" /[a-z. A-Z]23/ -- any a to z or uppercase A to Z followed by 23 /ab*/ -- a followed by 0 or more bs /[^>]+/ -- one or more non ">" /<td>(. +? )</td>/ -- start with a <td> tag, then "capture" as FEW characters as possible that you can end with a </td>

Don't Go There http: //stackoverflow. com/questions/1732348/regex -match-open-tags-except-xhtml-self-contained-tags http: //bit. ly/4 pvf. AB

Never parse HTML yourself from Beautiful. Soup import Beautiful. Soup as Soup from soupselect import urllib soup = Soup(urllib. urlopen('http: //slashdot. org/')) select(soup, '. tag-bar a')

SQL, no. SQL, memcache • For bigger projects • I dare you to find a IO-lab project that won't fit into my. SQL • But let's take 5 minutes to mock up a schema

• unit testing (devs like and are usually responsible for this one) • functional testing • system/integration testing • performance/security testing • regression testing • acceptance/verification testing (smoke tests)

Test Driven Development • literally write tests before you write the functionality • must confirm that a test fails before you write the code in order to make it pass • don't write any code unless you have a failing test (because that code may be unnecessary) • after all tests pass, you can refactor your code without fear because the tests will catch any issues you might introduce