An Introduction to Ubiquitous Computing Roy Want Intel

An Introduction to Ubiquitous Computing Roy Want Intel Corporation John Krumm Microsoft Corporation 2010

Outline 1. Founding Contributions to Ubiquitous Computing 1. 1 1. 2 1. 3 1. 4 Xerox PARC Tabs, Pads, and Liveboards Context Awareness IBM Research: Pervasive Computing versus Ubiquitous Computing 1. 5 University of Tokyo: T-Engine and the ITRON Operating System 1. 6 Hewlett Packard: Cooltown

Outline 2 Ubiquitous Computing in U. S. Universities 2. 1 UC Berkeley: Info. Pad 2. 2 MIT Media Laboratory: Wearable Computing 2. 3 Georgia Tech: Living Laboratories 3 Ubiquitous Computing in European Laboratories and Universities 3. 1 Olivetti Research: Active Badges 3. 2 Karlsruhe: Cups and Smart-Its 3. 3 Lancaster University: Guide 4 Modern Directions in Ubiquitous Computing 4. 1 Microsoft Research 4. 2 Intel Research

Outline The Research Community Embraces Ubiquitous Computing 6 The Future of Ubiquitous Computing 5

Computing Era 1. Mainframe computer ( 1940 s ~ ) n n n 2. Personal computer ( 1960 s ~ ) n 3. A single large time-sharing computer Owned by an organization Used by many people at the same time Owned and used by one person and dedicated to him. Ubiquitous computing (late 1980 s ~) n n Each persons and uses many computer Small networked portable computer products: smart phone, PDA, embedded computer

Founding Contributions to Ubiquitous Computing 1 2 3 4 Xerox PARC Tabs, Pads, and Liveboards Context Awareness IBM Research: Pervasive Computing versus Ubiquitous Computing 5 University of Tokyo: T-Engine and the ITRON Operating System 6 Hewlett Packard: Cooltown

Xerox PARC p Ubiquitous computing n p Ubiquitous computing was coined by Mark Weiser in 1988 at Xerox PARC, while serving as the director of the Computer Science Laboratory (CSL). Ubicom n Ubiquitous computing

Ubiquitous Computing The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it. —MARK WEISER 那些最深奧的技術都消失了，它們將自己融入 每天的日常生活直到它們無法從中區分。

The Essence of Weiser’s Vision p Mobile and embedded processors can communicate with each other and the surrounding infrastructure, seamlessly coordinating their operation to provide support for a wide variety of everyday work practices.

The Essence of Weiser’s Vision p. A consequence of Weiser’s vision Each device needs to limit the range of its communication n Enable valuable wireless bandwidth reuse n p In the early 1990 s, there were no short-range wireless standards p Today we have: n Bluetooth, Near Field Communication (NFC), Ir. DA, Zigbee, and Wi. Fi (soon Wi. Fi PAN)

Weiser’s Vision p Invisible computing n n p In a ubicomp world, computation could be integrated with common objects that you might already be using for everyday work practices, rather than forcing computation to be a separate activity. Ex: Read a printed page (printing technology) Calm technology n n ubicomp pushes the computerized versions of these technologies back into the physical world Ex: e-book -> real book

Tabs, Pads, and Liveboards PARC has long had a philosophy of “Build what you use, and use what you build” p The ubicomp research theme continued that tradition. p Liveboards p n p Pads n p Yard-scale: large and immovable Foot-sized: can be held in your hands and carried. Tabs n Inch-scale: can it in a pocket

Xerox Parc. Tab FIGURE 1. 3 (a) Xerox Parc. Tab a palm (inch-scale) computer communicating using diffuse infrared (IR) signaling (b) an infrared transceiver basestation installed in the ceiling of each room comprising the ubicomp environment. Note the ring of IR emitters at the edge of the circular board, and four IR detectors at the center pointing in four cardinal compass directions. (Photos courtesy of PARC, Inc. , http: //www. parc. com)

Xerox Parc. Pab

Xerox Liveboard FIGURE 1. 5 A commercial version of the Liveboard, sold by Liveworks in the mid 1990 s, which evolved out of the original wooden laboratory prototype developed at PARC (see Figure 1. 2). (Photo courtesy of PARC, Inc. , http: //www. parc. com)

Context Awareness p Allows applications to comprehend the environment in which they are being used, and adapt their operation to provide the best possible user experience.

Context Awareness A user or device context is difficult to model p It has many dimensions p n n n location, the identity of devices close by, who else is present, the time, environmental factors p n sound, motion, temperature, orientation, other physical variables, many of which can be measured through on-platform sensors.

Context Awareness p Context awareness can also span multiple levels of system architecture. p At the device level it can take advantage of on-board sensors. n p Example: Inverting a Parc. Tab would invert its screen At a higher-system level, applications could use context to modify their behavior. n Proximity Browser: Take advantage of the cache principle

Summary of ubicomp at PARC p It has taken ~15 years for n the underlying technologies to mature, n communication standards to be ratified, n and for many of the models to gain traction in the marketplace.

Summary of ubicomp at PARC This is the nature of ubiquitous computing. It is very sensitive to the affordances of the devices. p As technologies advance, technologists are trying to replace and, whether it be processor performance per watt, storage capacity, network bandwidth, display resolution, device size, and weight. p Each year more possibilities for the mainstream application of ubiquitous computing open up. p

IBM Research p In the mid-1990 s, IBM began a research direction it called pervasive computing (IBM Mobile and Pervasive Computing) p IBM, to its credit, was one of the first companies to investigate the business opportunity around pervasive systems, and created a business unit dedicated to the task.

IBM Research One of the first commercial deployments of a pervasive computing system was born from a collaboration between IBM Zurich and Swissair in 1999 (IBM Swissair), enabling passengers to check-in using Web-enabled (WAP) cell phones (Figure 1. 6).

IBM Research

IBM Research Once the passengers had accessed the service, the phone also served as a boarding pass, showing gate seat and flight departure information, and identifying the traveler as having valid fight credentials. Although this was one of the most publicized projects, IBM also applied these technologies to other service opportunities in banking and financial services, gaining early experience in this area

IBM Research One of the key enablers for pervasive solutions has been the development of wireless/mobile platforms running standard operating systems that are already widely deployed in the form of smart phones. p Web Sphere p J 9 Virtual Machine p

University of Tokyo T-Engine and the ITRON Operating System p In the late 1980 s, Prof. Ken Sakamura created the TRON research program at Tokyo University. p He developed a series of embedded computing platforms called T-Engines that were designed to be embedded in devices ranging in size from mobile electronics to home appliances and smart sensors p

T-Engine

University of Tokyo Sakamura developed ITRON, an embedded real-time operating system that was portable across the various scales of TEngine, and also able to run on a number of commercial platforms. p ITRON became very popular in Japan. p ITRON it was open source. p ITRON also led to incompatibilities in some cases, which limited the ability to share and run application code. p

Hewlett Packard: Cooltown The Cooltown project (Kindberg et al. , 2002) popularized the notion of linking real-world objects with Web content. p The key observation was that every object could have a Web page capable of describing it. p For non-electronic objects were by attaching electronic tags that encode a unique ID. p

Hewlett Packard: Cooltown In Cooltown, the tags were implemented with barcodes or IR beacons, and could contain either the full textual URL or a unique number that could be mapped to the URL in order to access the corresponding server. p A system that has access to the identity of all the people and objects in a particular place also has access to the defining context, and can make inferences about the activities taking place. p

Hewlett Packard: Cooltown p In addition to work and office situations, Cooltown also deployed its Web presence system in museums and other public exhibits to enhance user experience in that setting. p The applications of Cooltown provide us with another unique example of how a ubiquitous computing environment can be designed and deployed in the real world.

Ubiquitous Computing in U. S. Universities 1 UC Berkeley: n Info. Pad 2 MIT Media Laboratory: n Wearable Computing 3 Georgia Tech: n Living Laboratories

UC Berkeley: Info. Pad p p p The program was created by Prof. Bob Brodersen, involving a large graduate team in the mid 1990 s The objective of the Info. Pad program was to explore architecture and systems level issues for the design of a mobile computer providing ubiquitous access to real-time media in an indoor environment. and served as a vehicle for exploring individual system components and the low-power mechanisms needed to support effective mobility.

UC Berkeley: Info. Pad p The wireless communication architecture : n n p high data-rate downlink for display updates. 1 – 2 Mbps , in the 2. 4 GHz band low data-rate uplink for keyboard, mouse, and pen interaction. 121 kbps , 902 MHz The architecture of Info. Pad was highly tuned to the pad functionality, without requiring high-performance processors and memories, or mass storage devices

MIT Media Laboratory: Wearable Computing The wearable computing puts the emphasis on a portable computer that can be unobtrusively integrated with a person’s clothing, while still being comfortable for the wearer (Siewiorek et al. , 2008). p A key characteristic of such a device p n n the heads-up display integrated into a user’s glasses an input device that can be operated while out of view, typically in a pocket.

MIT Media Laboratory An important related topic is augmented reality (Feiner et al. , 1997) p Augmented Reality is able to overlay information on top of what a user sees in order to improve ability to carry out a task. p n For example, a maintenance engineer servicing an engine would benefit from a visual overlay of labels that identify key components augmenting what she would normally see.

Georgia Tech: Living Laboratories p In the late 1990 s, the Future Computing Environments group at Georgia Tech (Atlanta, GA) began a number of research activities under the general category of Living Laboratories. Prof. Gregory Abowd is the lead researcher and cofounder of the group. p The best known for two exploratory ubicomp projects: Classroom 2000 n Aware Home n

Classroom 2000 began in July 1995 with a bold vision of how ubiquitous computing could be applied to education and provide added value to standard teaching practices in the future (Figure 1. 11). p Classroom 2000 investigated the possibility of capturing the entire lesson in a form that would be a useful reference itself. (the Xerox Liveboard was used for some of this work) p

Classroom 2000 has proven its value through popular demand, and was replicated five times on the Georgia Tech campus, and further deployed at three other universities. p Classroom 2000 is an excellent example of ubicomp being applied to a real problem, and creating a solution of measurable value. p

Aware Home In 1999, the Aware Home project was founded and set out to explore how computation and embedded echnologies could support everyday activities in a home. p The building was completed in 2000 (Figure 1. 12), and represents one of the largest dedicated spaces created to explore the vision of ubiquitous computing — a full embodiment of Weiser’s ideas of 10 years earlier. p

Aware Home p Systems introduced into the Aware Home to support the research n n included cameras and RFID tags to identify and track an occupant’s location, and various forms of sensors. For example, the house included a smart floor composed of a network of pressure sensors that could identify the characteristic ambulatory gait of individuals as they moved between rooms, thus providing additional means of occupant identification.

Aware Home Researchers found that characteristic signals in the consumption of power, gas, and water over time could be used to accurately infer and model activities going on in the house. p The Aware Home also investigated techniques to allow elderly residents to remain in their homes. p

Ubiquitous Computing in European Laboratories and Universities 1 Olivetti Research: Active Badges 2 Karlsruhe: Cups and Smart-Its 3 Lancaster University: Guide

Olivetti Research: Active badges About the same time as Xerox PARC embarked on its ubiquitous computing program, Olivetti formed a research laboratory in Cambridge, UK. p They focus on high-speed local area networks (LANs), and the novel applications and systems. p

Olivetti Research: Active badges p p Toward this goal, the Pandora project was formed to explore the design of integrated multimedia services based on a Cambridge Fast Ring (a 100 Mbps slotted ring: the slot passing mechanism providing an inherently fair scheme for sharing bandwidth among multiple network clients). To support integrated services, which included audio and video traic (note that this is 1988, the age of Plain Old Telephone Service, and well before Skype and cell phones), there was interest in using distributed computation to provide sophisticated telephone services such as video telephony and conferencing using a LAN.

Olivetti Research: Active badges The Pandora system, a dial-by-name approach was adopted. p The system still lacked the ability to deliver a call to the location p To solve this problem, the Active Badge Location System was created (Figure 1. 13). p

Olivetti Research: Active badges The Active Badge Location System was the first automated indoor location system p It used a diffuse infrared beacon embedded into the electronic badges (Active Badges) worn by various members of the laboratory (Want et al. , 1992). p Each badge emitted an infrared signal encoding a unique ID that was received by a network of sensors, typically one sensor per room (multiple for large conference rooms). p

Olivetti Research: Active badges The Active Badge system provided one of the earliest available location systems and opened up the possibility of experimenting more generally with location-based services. p As a result, Olivetti received many inquiries about the system, and built a consignment of badges and sensors to supply research laboratories around the world p n Xerox PARC, Euro. PARC, DEC SRC, DEC WRL, MIT Media Laboratory, Cambridge University Computer Laboratory.

Olivetti Research: Active badges Olivetti continued to be a leader in developing novel location technologies, and by 1997 designed the Active BAT system, with the goal of providing accurate threedimensional (3 -D) positioning within a room. p This capability was achieved with a new type of badge (BAT) that combined ultrasound and radio to enable range measurements to be made from an array of sensors placed in the ceiling of every room. p

Olivetti Research: Active badges MIT produced an alternate indoor positioning system shortly after Olivetti. p This system was also based on ultrasound, but used a design approach that was the inverse of the BAT system. p MIT system had some privacy advantage over the BAT system, as each mobile could then decide who else could ind out its location p But MIT system can decide the accurate 3 D location p

Karlsruhe: Cups and Smart-Its One of the strongest university ubiquitous computing programs in continental Europe is found at the University of Karlsruhe (Germany), also the origin of the Ubicomp conference. p In 1997, researchers began experimenting with ubicomp concepts and gained experience embedding computing into everyday objects, the Media. Cup being a well-known example (Figure 1. 14 a). p

Smart-Its Karlsruhe led the formation of the Smart. Its project, a collaboration with ETH (Zurich) and Lancaster (UK), Viktoria Institute (Sweden), Interaction Institute (Sweden), and VTT (Finland). p The goal of the Smart-Its project was to create a general-purpose tool for experimenting with ubicomp concepts and providing researchers with a hardware and a software development environment well suited to ubicomp research (www. smart-its. org). p

Smart-Its are small embedded computers with communication and sensing components that can be further integrated with everyday objects such as furniture, clothing, coffee cups, product packaging, and fresh produce (Figure 1. 14 b). p The initial research collaboration resulted in 16 different projects exploring a wide variety of applications. p Smart-Its have been a very successful research platform and widely adopted across the European university network. p

Lancaster University: Guide Since the mid-1990 s Lancaster University has been a leading university research center in England contributing to the ubicomp vision. p From 1997 to 2001, the GUIDE project was created, obtained a government grant, and captured the imagination of many researchers interested in locationbased services in the wild. p

Lancaster University: Guide It was the first mobile electronic guidebook designed and optimized from concept to implementation for use by tourists. p The user platform was based around a rugged Fujitsu Team. Pad tablet p The system also used a mobile tablet with a display small enough to carry, but large enough for effective user interaction, and it had the ability to add new information on the fly. p

Lancaster University: Guide p p it used NCR Wavelan radios compatible with the 802. 11 b standard (new at that time), and took advantage of the characteristic poor radio propagation to create servers that covered a region called a cell. Each cell server presented the user with options to learn more about areas of interest. Once the user made a selection, a Web page was retrieved from the GUIDE Web server providing a full description of nearby points of interest. GUIDE was the first electronic tour guide that made use of a microcellular communication network to determine location in a citywide deployment.

Modern Directions in Ubiquitous Computing 1 Microsoft Research n 2 Microsoft Research (MSR) was founded in 1991 in Redmond, WA, with expansion to Cambridge, England, in 1997, and Asia in 1998. MSR Bangalore (India), Silicon Valley, and Cambridge, MA, were later established between 2005 and 2007. Intel Research n n In 1990, Intel established a new organization called Intel Research to begin of-roadmap exploratory research. it founded a number of laboratories in close proximity to universities with top-tier computer science departments

Microsoft Research (MSR) Sense. Cam & My. Life. Bits Sense. Cam is a small wearable computer that periodically captures images of the world as a user moves around. p My. Life. Bits project, Sense. Cam provides a wealth of contextual data about the wearer, augmented by a database describing documents and other electronic media that the individual has accessed. p

Microsoft Research (MSR) p RADAR n n the first example of a wireless system allowing mobile computers to locate themselves in a building an indoor global positioning system (GPS) RSSI : the received signal strength indication accuracy of 2– 3 meters

Microsoft Research (MSR) p Easy. Living n n n established in 1999 MSR’s closest project to the spirit of the original ubicomp vision. smart room: to support both work and recreational activities.

Intel Research Seattle (IRS) Place Lab was the best known of the projects from IRS, exploring and building a system that could determine the location of a mobile device by cataloguing and mapping Wi. Fi access points throughout a city. p This was later extended to GSM towers, and demonstrated effective location-based services for cell phone applications p Today, this approach has been adopted by several companies, especially the Apple i. Phone. p

Intel Research Berkeley p p various research projects were created around sensor networks and Motes a small form factor wireless sensor node that was the heart of these systems. Sensor networks share a common thread with the embedded computation vision of ubicomp. Although they are usually associated with scientific monitoring, they can also be integrated with objects in the environment to provide embedded sensing and control (similar to the Smart-Its work).

Intel Research p The Personal Server project in 2001 took a fresh look at the trends in computation and mobility, recognizing that even handheld mobile computers were beginning to make use of highperformance processors along with high -capacity solid-state memories, and with low-power dissipation they could be operated from a small battery.

Intel Research p The Personal Server project reversed the traditional design paradigm, and instead of using the mobile device as a client for a remote network server, it used the mobile device as the server and ran the client on a nearby PC, thus enabling a user to gain access through a large high-resolution display and full-feature keyboard.

Intel Research p The personal server concept was initially implemented on a custom XScale embedded system called Stargate, and later ran on a Motorola e 680 cell phone (Linux/XScale), demonstrating that one day the ubiquitous cell phones could become our primary computer, which we would interact with wirelessly through nearby infrastructural computers.

The Research Community Embraces Ubiquitous Computing Journals and magazines: p IEEE Personal Communications, 1994 p Personal & Ubiquitous Computing, 1997 p IEEE Pervasive Computing, 2001 p n p IEEE Wireless Communications, 2001 n p systems and applications level The lower layers of the protocol stack IEEE Transactions on Mobile Computing, 2002

The Future of Ubiquitous Computing p Since 2000, there has been a dramatic improvement in the capabilities of highend cell phones, also known as smart phones. p Not surprisingly, the cell phone has become the most ubiquitous computer there has ever been.

The Future of Ubiquitous Computing p p To understand where this market is going, we look at a new family of mobile processors called Atom, which was announced at the Intel Developers Forum in Shanghai, China, in April 2008. The first generation of processors in this family is based on a 45 nm process, is a fully compatible × 86 instruction set, and is designed for lowpower operation in the 1– 2 W range. The primary advantages of Atom are performance and software compatibility. Virtualization techniques

The Future of Ubiquitous Computing p p By using virtualization techniques, application code running on a PC in a Virtual Machine (e. g. , KVM) can be migrated across a wireless connection to a mobile device, and then back to a PC to facilitate both mobile and enterprise work practices—again, all without modifying the applications. Uniformity of processor architecture and the opportunity for a mobile runtime environment that is compatible with the desktop, server, and embedded market will accelerate the development of ubicomp applications and facilitate their use in many new situations.

The Future of Ubiquitous Computing p Just as the PC provided a common platform for the successful evelopment of widely available desktop software in the early 1980 s, cell phones are poised to take that role in the next decade. p Weiser’s vision n Smart phone -> tab Laptop -> Pad Large LCD -> liveboard

The Future of Ubiquitous Computing p At this time, p ubicomp is no longer just a vision; p it is rapidly becoming the reality.

An Introduction to Ubiquitous Computing End