Enhancing the Web With EndUser Programming Tak Yeon

Enhancing the Web With End-User Programming Tak Yeon Lee, Ben Bederson

Outline Problem Inefficiencies on the Web Challenge Building extension is very hard Our goal Expressive and Easy Programming Env. - Improves a wide range of tasks - No programming skills required

Snapshot ANY WEBSITE

Inefficiencies on the Web 1. Missing relevant information “Is it a good value? ” Kelly’s Blue book “Is the model reliable? ” Auto forums “I want to see the pictures. ” Detail page “How far is this place? ” Google Map

2. Removing unwanted information AD AD

2. Removing unwanted information

3. Repetitive Operations “I want to download all PDFs named as [first author]-[title]-[year]. pdf ” 10. 1. 1. 154. 444. pdf Howe-The. Rise. Of. Crowdsourcing-2006. pdf Kittur-08 -001. pdf Kittur-Crowdsourcing. User. Studies-2008. pdf 60947(1). pdf Brabham-Crowdsourcing. As. AModel. For-2008. pdf Alonso-Crowdsourcing. For. Relevance. Evaluation-2008. pdf

4. Finding a needle in a haystack Grading Procedure 1. Finding mistakes Takes 4 -8 hours to grade 70 students 2. Add comments 3. Calculate the total points 4. Submit to grade server

Summary of Potential Mistakes Grading with an extension My. Double. zero 2. Click to add predefined comment My. Double. zero 1. Click to scroll to the code line of the mistake “Previously it took several hours, but now I can finish in 40 minutes. ” - Lecturer “Three times faster than before, at least!” 3. Setting partial -Grading TA credit My. Double. zero 4. Report total score to another server

Browser Extensions add new features and personalize experience. Over 50, 000 on Chrome Web Store Ad. Block Photo Zoom Print All >15 million users >2 million users E-mails in a folder Building an extension requires • Programming knowledge (HTML, CSS, and Ajax) • Time and effort of development • The grading example took >40 hours

OUR GOAL IS TO BUILD Expressive and Easy programming env. The toolkit should support tasks like. . . • • Automating repetitive jobs Integrating data from multiple sources Custom filtering Information Summary We defined a domain language model having Pick, Change, Add operations as generic functionalities of the extensions.

Pick Change Add model Pick extract information from web pages or elements. <A> tags The rise of crowdsourcing. pdf Crowdsourcing user studies. . . Crowdsourcing as a model. . . URLs http: //sistemashumano. com. . . http: //dl. acm. org/citation. cfm ? id=1357127 http: //con. sagepub. com/. . .

Pick Change Add model Change manipulates dataset. [b] 3 [a] 1 • Arithmetic operations • +, -, /, %, . . . • sum, avg, count, . . . 2 3 • String manipulation [a] * [b] 3 6 9 • concatenate, split, find, replace, . . . • Filter • Sort

Pick Change Add model Change manipulates dataset. [b] - [a] I • Arithmetic operations • +, -, /, %, . . . • sum, avg, count, . . . am a boy • String manipulation Join [a] with [b] I-am a-boy • join, split, find, replace, . . . • Filter • Sort

Pick Change Add model Change manipulates dataset. [a] a. jpg [b] png • Arithmetic operations • +, -, /, %, . . . • sum, avg, count, . . . b. gif c. png • String manipulation Filter [a] having [b] c. png • join, split, find, replace, . . . • Filter • Sort

Pick Change Add model Change manipulates dataset. [a] Toronto • Arithmetic operations • +, -, /, %, . . . • sum, avg, count, . . . Seoul Paris • String manipulation Sort [a] Paris Seoul Toronto • join, split, find, replace, . . . • Filter • Sort

Pick Change Add model Add modify the original web pages. • Create new elements • image, text, button, . . . • Modify element’s style • Show / Hide, changing color, fonts, . . . • Triggering events • click, . . . • Type text

Pick Change Add model e. g. Removing AD banners [1]. Pick ad banner elements [2]. Hide [1] AD AD

Pick Change Add model e. g. Downloading files with custom name pattern [1]. Pick title text The rise of crowdsourcing. pdf [2]. Merge [1] with name pattern “. pdf” Crowdsourcing user studies with Mechanical Turk. pdf [3]. Pick download links Crowdsourcing as a model for. pdf [4]. Add [2] to ‘download’ attribute of [3] Crowdsourcing for relevance evaluation. pdf

OUR GOAL IS TO BUILD Expressive and Easy programming env. • Requires no programming knowledge • Programming-by-Example techniques • Telling computer what I want, without worrying how to do.

Programming-by-Example technique INPUT PROGRAM OUTPUT Traditional Programming-by-Example ? 1, 2, 3 int sum=0; for(int v in input) { sum=sum+v; } return sum; ? ! 6

Programming-by-Example technique Programs with multiple inputs e. g. Adding suffix to strings [1] [2] Paris Input . jpg New. York Seoul Tokyo Merge [1] with [2] Paris. jpg Output New. York. jpg Seoul. jpg Tokyo. jpg

Programming-by-Example technique Multi-step programming e. g. Counting items of [1] that exist in [2] [1] [2] Paris, France Korea Istanbul, Turkey Japan Seoul, Korea Osaka, Japan Filter [1] exist in [2] [3] Seoul, Korea Tokyo, Japan Count [3] 2

Programming-by-Example technique Multi-step programming e. g. Counting items of [1] that exist in [2] [1] [2] London, UK UK LA, US Canada Beizing, China Toronto, Canada Filter [1] exist in [2] [3] London, UK Beizing, China Toronto, Canada Count [3] 3

Demo time

Conclusion A new opportunity for end-users to personalise the web by creating their own extensions. Programming-by-Example techniques with Pick, Change, Add model Future work includes • Usability Tune up • Evaluation of the system’s learnability Please email us if you have an idea for extensions Tak Yeon Lee <tylee@umd. edu>