Getting a dozen 20 year olds to work

Getting a dozen 20 -year olds to work together for fun and. . . social good

Overview and outline Team of undergraduates E-Mission + = ? ? ?

E-mission project Background and motivation

E-mission Motivation 2050 2035 2020 Baseline 0 50

E-mission Overview

E-mission research questions �Open source travel diary collection (M) �Extension to survey data collection (P) �Aggregate modelling and analysis (A) If you have a project in one of these areas, come talk to me! Happy to deploy (with adequate support) and generate results.

IT for sustainability Motivation, overview and structure

Class motivation (societal) �Scale and complexity of problems is large. Lots of work is interdisciplinary, which makes it hard because you have to deal with non core CS aspects (I say so!) �Need for long-term and sustained system building to tackle large-scale social problems (David Patterson says so!) �Many CS students have programming experience when entering. How to engage them, provide challenging work and deepen skills? (David Corman says so!)

Class motivation (individual) �Researcher � Help with routine tasks � Skills that researcher does not have (e. g. UI design) � Interdisciplinary skills (e. g. behavioral econ) � Larger pool of people to avoid design bias � Throwing away final projects year after year seems wasteful �Students � Build something real and end-to-end that is used after class is done � Learn to identify and split up real-world problems � If publishable, improve grad school admission chances

Class setup � 15 students max. 12 finally enrolled �CS majors, but outside minors related to behavior, transportation or sustainability a plus � Made an exception for a business major, didn’t work so well �Grades in lower division courses � Made an exception for a B student really interested in sustainable transportation, worked pretty well

Class structure � First half, skill building. In 2 -3 person teams, fix a bug related to the skill of the week. � Phone based sensing and interaction � Service architecture and backend � People � Data integration � Exploratory Data Analytics � Second half, form small project teams and work on three related projects that contribute to e-mission (their choice) � Tour models � User models and recommendations � UI display and gamification

ITS class Evaluation and lessons learned

Class evaluation (what worked) �Experience of planning for class was great � Forced lit review � Forced articulation of ideas to group of non-experts � Forced cleaning up code to allow others to run it � Forced improved documentation �Students implemented a recommendation system that worked end to end � Divided up work into teams � User model, tour model, alternatives, UI � Everything “worked” together at the end (with a little help) �Class evaluation was positive (6. 6/7 overall)

Trip clustering for tour models (Zackery Field, Jeff Huang, Gautham Kesineni) Clustering trips to generate tour models (needs to work for both short and long)

Recommendation system (Shaun Singh, Josh Zarrabi, Jimmy Diefenderfer, Shanthi Shanmugam)

UI and games (Neeraj Baid, Lesley Chiang, Jessica Wong, Handson Wu)

Class evaluation (what didn’t) �Ideas are good, but quality is low (demo/prototype) � Recommendations are not necessarily meaningful � Evaluation? We don’t need no stinkin’ evaluation �Why? � Shared global state and read after write dependencies �State now distributed across project groups �Student B depends on Student A’s code, but Student A has a midterm this week �Student B depends on Student A’s code, but Student A writes crappy, ill-tested code. Now Student B has to play janitor.

Lessons learned �Undergraduate students cannot be expected to write serious infrastructure � “Testing takes too much time” in spite of many efforts � Hacky code to “get things to work” �Prepare to do a ton of “glue” work to get the pieces to work together � Integrate early and often to force them to do more �Don’t make students build complex systems with lots of dependencies � Much easier to work on small, bite-size pieces that can be scheduled around other commitments

What would I change? � Build the infrastructure and evaluation system first � One example that can be copied � Dashboard/autograder/evaluation system � Contests! � Focus around multiple alternatives to the example � Run contests for greater motivation � Require deployment for evaluation � Similar to kaggle, but with ongoing data collection � Errors = bad results = bad score Will try this out in Spring or Fall 2016 and report back…

Questions? I’d love to hear from you if you have tried something like this.