Bootstrapping Ubiquitous Computing Jason I Hong jasonhcs cmu

Bootstrapping Ubiquitous Computing Jason I. Hong jasonh@cs. cmu. edu

Ubiquitous Computing is Coming • Advances in wireless networking, sensors, devices – Greater awareness of and interaction with physical world • Ubicomp can help in coordination, efficiency, safety Find Friends E 911 Incident Command

Four Key Problems Facing Ubicomp • Requires very high level of expertise to build – Sensors, distributed systems, security, interaction… • Difficult to deploy and realistically evaluate – Brittle, hard to get out of lab, small evaluations • Privacy is a high barrier to acceptance • Few compelling ubicomp apps – Major criticism of field, often tech push, limited in utility – Hard to “take it to the next level” & make real impact!

Research Directions • Requires very high level of expertise to build – What if creating ubicomp apps were as easy as HTML? • Difficult to deploy and realistically evaluate – Can we provide tools for rapid prototyping? – Better and more robust infrastructure? • Privacy is a high barrier to acceptance – Infrastructure biased towards privacy? – Better tools and methods for creating? • Few compelling ubicomp apps – Some of my ideas…

Topiary A Prototyping Tool for Location Enhanced Apps • • Quickly create mockups of location-based apps “Run” these mockups with real users to get feedback

Place. Lab for Acquiring Location • Place. Lab location via local database of Wi. Fi Points – Unique Wi. Fi MAC Address -> Latitude, Longitude – Periodically update your local copy A –Works indoors and C in urban canyons –Works with encrypted nodes B –No special equipment –Privacy-sensitive –Rides the Wi. Fi wave

Place. Lab SF Bay Area ~60000 Nodes (~4 Megs)

Place. Lab f o y t i s r e v i y Un e l e k r e B a i n r o f Cali Berkeley Campus ~1000 Nodes

Topiary Extending this Work • Model context beyond location – Light level, motion, number of people around, etc – How to make it easy to mockup and quickly test? • Push it by trying to model other kinds of ubicomp apps – Ex. Non-visual output • Topiary as a authoring tool for ubicomp apps? – Like a web page authoring tool – People could deploy ubicomp apps just by deploying special kinds of web pages (Context. HTML)

Place. Lab Extending this Work • Moving from “locations” to “places” – – – – • Parents -> “busy” Friend -> “at school” Officemate -> “in Newell Simon 3002” Spouse -> “I’ll be at the café in 5 minutes” Roommate -> “I’m eating” Diary study or cell phone study to answer “where are you? ” Relation to inquirer, distance, familiarity with area Moving from “locations” to “events” – You were in NSH 3002 at 1: 35 PM Aug 26 2004 – Google shows that this was where Jason was speaking – Imagine linking this to personal photos taken. . .

Research Directions • Requires very high level of expertise to build • Difficult to deploy and realistically evaluate • Privacy is a high barrier to acceptance • Few compelling ubicomp apps

End-User Programming of Ubicomp • Conjecture: Events natural way of perceiving the world – My alarm clock is ringing – The dispenser is now out of soap – The washing machine has finished • Idea: Use events to create simple ubicomp apps – When alarm clock rings – When out of soap – When laundry done • -> start coffee maker -> send email to maintenance -> send SMS to me Make it easy for non-CS frosh to create useful apps

End-User Programming Starting this Work • Lo-fi prototypes – Are events right? What kinds of events most useful? – What kinds of outputs most useful? – Metaphors? Like calendar events on Palm? Like VCR? • Systems Issues – Security? Who has access to my alarm clock? – How to actually send the events? – How to actually understand what’s going on? • Could use same infrastructure as Context. HTML

Research Directions • Requires very high level of expertise to build • Difficult to deploy and realistically evaluate • Privacy is a high barrier to acceptance • Few compelling ubicomp apps

Privacy • Most often-cited criticism of ubicomp • My dissertation work – Interviews, surveys to understand privacy concerns – System architecture (capture, store, process locally) – User interface widgets • Continuing work – Privacy risk models (like task analysis for privacy) – Design patterns for ubicomp

Privacy Extending this Work • • Privacy at the physical layer? Extending the risk model Continuing the design patterns Deployment and eval of apps – How do perceptions change over time? – How do people make decisions about what info to share? – Better architectures?

Research Directions • Requires very high level of expertise to build • Difficult to deploy and realistically evaluate • Privacy is a high barrier to acceptance • Few compelling ubicomp apps

Applications • • • Location-enhanced Instant Messenger Eventster Pokedex

Location-enhanced Instant Messenger

Location-enhanced Instant Messenger

Extending this Instant Messenger • Multiple modes of input – Pictures, photos (think camera phone), sketches, voice, text – Make it even easier to coordinate • Use contextual information to streamline replies – “Where are you? ” – “I am in…” • (1) school • (2) Physics class • (3) Pittsburgh • How effective is the privacy here? – Even simpler UI for managing privacy? – Deploy and eval here at CMU

Eventster • Location-based matchmaking – “You were the cute redhead in the Safeway line in front of me this morning. Coffee? ” – “Anybody see the car accident at this time and place? ” – “Garage sale this Saturday at 5 th and Morewood” • Personal location history – Devices automatically store your location history locally – Can retrieve and process queries privately • Download Eventster streams – Find ones that match your past locations – “What’s going on near me now? ” • Screen scrape classifieds, CMU talk pages, Craigslist

Real-World Pokedex • Discover the beauty of the world around us – “What kind of bird is this? ” – “Is this a maple tree? ” • New interaction techniques – Combine location info, computer vision, and 20 questions – Present most likely choices to users to mediate

Testbed for Ubicomp • What if all students were given a wifi camera phone? – Combines computation, communication, cameras, location – Large testbed and large #users, realistic evaluations • Hard systems issues – How to scale in #users and area? – How to make it easy to program? Maintain? • Hard HCI issues – – Can we strengthen local communities? How to design for privacy? New interaction techniques for mobile devices? How to help people manage data?

Summary • Address four key problems in ubicomp – – • Requires very high level of expertise to build Difficult to deploy and realistically evaluate Privacy is a high barrier to acceptance Few compelling ubicomp apps Ubicomp Course this Fall – MW 9 -10: 30 AM in NSH 3002 (this room) • Send mail to jasonh@cs. cmu. edu – Or drop by NSH 3613

Bootstrapping – Initial Apps Other Ideas • Games – CMU scavenger hunt • Stitching pictorial directions together – Use photos people take to create map directions • Auto. Diary – “Where was that restaurant I was at last month? ” • Mo. Blogs – Extremely easy content authoring • • Dude, where’s my car? VOIP – Free Wi. Fi VOIP on campus

Bootstrapping – Initial Apps Lots of Potential Here! • • Combine computation, communication, cameras, location Can address hard systems issues – – • How to scale in #users and area? How to make it easy to program? How to make it easy to maintain? Machine learning to “learn” places? (“Home”, “Cool place”) Can address hard HCI issues – – Can we strengthen local communities? How to design for privacy? New interaction techniques for mobile devices? How to help people manage data?

Outline Compelling Ubicomp Apps Tools & Infrastructure for Development, Deployment, Eval Deployment Testbed across all of CMU / Pittsburgh

Bootstrapping – Tools • • • Rapid Prototyping Programming Support Infrastructure Support Give what kinds of

Bootstrapping – Tools Rapid Prototyping • Goal: help people generate and iterate on ideas faster • Privacy Risk Models for Ubicomp Design Patterns for Ubicomp •

Bootstrapping – Tools Rapid Prototyping • Topiary

Bootstrapping – Tools Rapid Prototyping • Interesting research questions / ideas… – Extending Topiary beyond location prototyping • Activity, orientation, busy, … – Improving Wizard of Oz techniques • Tradeoff between design effort and wizard effort – Scaling for large numbers of ppl • MMORPGs as a prototyping platform? • Anonymized traces of user behavior to simulate? – Tools to prototype sample applications? • Location IM, Location web, Medic. Alert++, Eventster, Pokedex

Bootstrapping – Tools Programming Support • Programming ubicomp? – Topiary real-world compiler – Confab toolkit for privacy-sensitive ubicomp • Scripting for ubicomp – Like HTML for ubicomp • Tim Berners-Lee didn’t invent amazon, shutterfly, ebay, blogs, but he did invent HTTP, HTML, and URLs.

Bootstrapping – Tools Infrastructure Support • Core services and infrastructure that would make applications easier to build and more robust? – Basic device services? Network? Building? • Cheap, robust, privacy-sensitive architectures? – Other things like Place. Lab? – Occasionally connected computing? – Can we learn anything from biology? • Self-org, (usually) robust to failure, billions of cells

Bootstrapping – Tools Infrastructure Support • Ubicomp platforms? – Cheap, simple, extensible, wireless sensor platform you can just plug into the wall (unlimited power) – Make it so simple and so useful… • we could use it in every room here at CMU • we could get freshman to use it – Easy end-user linking to create more powerful services • Alarm clock rings -> start coffee maker • Out of soap -> send email to maintenance • Laundry done -> send SMS to me • Sort of like creating a calendar event

Outline Compelling Ubicomp Apps Tools & Infrastructure for Development, Deployment, Eval Deployment Testbed across all of CMU / Pittsburgh

Bootstrapping – Testbed • Can we convince people to carry another gadget? – Possibly one that supercedes their cell phone? • Tools to help run user tests? Field studies? • Tools formative evaluations? – Put periodic surveys on phones • Tools for logging and analysis? – Get anonymized usage data from wifi phones • Better ways for evaluating for privacy? – Useful metrics for privacy? – Better user studies for privacy?

Advancing Research • Lots of research issues here Better user interfaces to manage privacy? Larger scale evaluation of privacy Privacy GUI widgets Updating student phones w/ new software • Business model for this? • Lots of student involvement here • • – Get students involved in brainstorming sessions – Do field studies identifying hidden needs

Matrix • Rows of Tools Testbed Apps Cols of themes • Come back to problems slide •

Special Topics in Ubicomp

Vision of Ubiquitous Computing We will reach a point where the combination of powerful processors, limitless data-storage capacity, ubiquitous sensor networks, and deeply embedded user interfaces will create a bond between human and machine “so intimate that users may reasonably be considered superhumanly intelligent. ” - Vernor Vinge

Backup Slides

Observation Chicken and Egg Problem • • Few compelling apps because… Apps are hard to build and deploy because… Few tools and little infrastructure because… Few compelling apps…

High-Level Strategy Large-scale Bootstrapping of Ubicomp Compelling Ubicomp Apps Tools & Infrastructure Deployment Testbed • Idea: address these issues simultaneously • Create tools to greatly improve ability to develop, deploy, and eval apps and testbed • Create a testbed of students and faculty to facilitate development,

Basic Bootstrapping Approach Start Simple and Incrementally Grow • Bootstrapping Round 1 – Brainstorm and build initial set of applications (app pull) – Brainstorm and build initial set of tools (tech push) • Don’t have to solve all of ubicomp, just useful subsets • Make it easy for frosh to create and deploy cool apps – Provide initial testbed platform by giving devices to HCII • Location-enabled Wi. Fi camera phones • Provide hooks for automatic updates and remote eval • Bootstrapping round 2+ – Apps inform tools and infrastructure (and vice versa) – Slowly and continually grow testbed until all of CMU – Continually evaluate apps

Why Bootstrapping? When all else fails, steal someone else’s ideas • Engelbart – Created tools bootstrapping how they interacted with computers • Berkeley CAD – Created design tools bootstrapping how they created microchips • Results – – Faster turnaround times (continually improve ability to create) Realistic usage by team and by others Higher quality tools Led to better research and greater impact

Outline Compelling Ubicomp Apps Tools & Infrastructure for Development, Deployment, Eval Deployment Testbed across all of CMU / Pittsburgh

Bootstrapping – Initial Apps Web Page Location Auto-Fill • Auto-fill location info on web sites Page. Modification URL =http: //www. starbucks. com/ txt. City =City. Name txt. State =Region. Code txt. Zip =ZIPCode Map. Quest Starbucks

Bootstrapping – Initial Apps Emergency Response • • Improve effectiveness of responders Medic. Alert++ location support – Location info stored by third-party – Firefighters can see ppl in building if needed