P 2 P Networking What is peertopeer P
- Slides: 85
P 2 P Networking
What is peer-to-peer (P 2 P)? n “Peer-to-peer is a way of structuring distributed applications such that the individual nodes have symmetric roles. Rather than being divided into clients and servers each with quite distinct roles, in P 2 P applications a node may act as both a client and a server. ” -- Charter of Peer-to-peer Research Group, IETF/IRTF, June 24, 2004 2/85 (http: //www. irtf. org/charters/p 2 prg. html)
Client/Server Architecture GET /index. html HTTP/1. 0 Client HTTP/1. 1 200 OK. . . Server 3/85
Disadvantages of C/S Architecture Single point of failure n Strong expensive server n Dedicated maintenance (a sysadmin) n Not scalable - more users, more servers n 4/85
The Client Side Today’s clients can perform more roles than just forwarding users requests n Today’s clients have: n ¨ more computing power ¨ more storage space n Thin client Fat client 5/85
Evolution at the Client Side DEC’S VT 100 No storage ‘ 70 IBM PC @ 4. 77 MHz 360 k diskettes ‘ 80 A PC @ 4 GHz 100 GB HD 2007 6/85
What Else Has Changed? n The number of home PCs is increasing rapidly Most of the PCs are “fat clients” As the Internet usage grow, more and more PCs are connecting to the global net Most of the time PCs are idle n How can we use all this? n n n 7/85
Resources Sharing n What can we share? ¨ Computer n resources Shareable computer resources: ¨ CPU cycles ¨ Data ¨ Bandwidth ¨ Storage Space - seti@home, GIMPS - Napster, Gnutella - PPLive, PPStream - Ocean. Store, CFS, PAST 8/85
SETI@Home SETI – Search for Extra-Terrestrial Intelligence n @Home – On your own computer n A radio telescope in Puerto Rico scans the sky for radio signals n Fills a DAT tape of 35 GB in 15 hours n That data have to be analyzed n 9/85
SETI@Home (cont. ) The problem – analyzing the data requires a huge amount of computation n Even a supercomputer cannot finish the task on its own n Accessing a supercomputer is expensive n What can be done? n 10/85
SETI@Home (cont. ) n Can we use distributed computing? ¨ YEAH n Fortunately, the problem can be solved in parallel - examples: ¨ Analyzing different parts of the sky ¨ Analyzing different frequencies ¨ Analyzing different time slices 11/85
SETI@Home (cont. ) The data can be divided into small segments n A PC is capable of analyzing a segment in a reasonable amount of time n An enthusiastic UFO searcher will lend his spare CPU cycles for the computation n ¨ When? Screensavers 12/85
SETI@Home - Example 13/85
SETI@Home - Summary n SETI reverses the C/S model ¨ Clients can also provide services ¨ Servers can be weaker, used mainly for storage n Distributed peers serving the center ¨ Not n yet P 2 P but we’re close Outcome - great results: ¨ Thousands of unused CPU hours tamed for the mission ¨ 3+ millions of users 14/85
15/85
16/85
17/85
Google -- Larry Page and Sergey Brin 18/30
Cloud computing 19/30
MUREX: A Mutable Replica Control Scheme for Peer-to-Peer Storage Systems
Murex: Basic Concept Hot. OS Attendee
Peer-to-Peer Video Streaming Video stream … …
Peer-to-Peer Video Streaming
Napster -- Shawn Fanning 24/85
25/85
History of Napster (1/2) 5/99: Shawn Fanning (freshman, Northeastern University) founds Napster Online (supported by Groove) n 12/99: First lawsuit n 3/00: 25% Univ. of Wisconsin traffic on Napster n 26/85
History of Napster (2/2) n 2000: estimated 23 M users 1984~2000, 23 M domain names are counted vs. 16 months, 23 M Napster-style names are registered at Napster 7/01: simultaneous online users 160 K n 6/02: file bankrupt n… n 10/03: Napster 2 (Supported by Roxio) (users should pay $9. 99/month) n 27/85
Napster Sharing Style: hybrid center+edge Title song 1. mp 3 song 2. mp 3 song 3. mp 3 song 4. mp 3 song 5. mp 3 song 6. mp 3 song 7. mp 3 1. Users launch Napster and connect to Napster server 2. Napster creates dynamic directory from users’ personal. mp 3 libraries 3. beastieboy User Speed beasiteboy DSL kingrook T 1 slashdot 28. 8 enters search criteria s ong 5 4. Napster displays matches to beastieboy 5. makes direct connection to kingrook for file transfer beastieboy “beastieboy” • song 1. mp 3 • song 2. mp 3 • song 3. mp 3 • song 5. mp 3 “kingrook” • song 4. mp 3 • song 5. mp 3 • song 6. mp 3 “slashdot” • song 5. mp 3 • song 6. mp 3 • song 7. mp 3 28/85
Gnutella History n n n n Gnutella was written by Justin Frankel, the 21 -year-old founder of Nullsoft. (Nullsoft acquired by AOL, June 1999) Nullsoft (maker of Win. Amp) posted Gnutella on the Web, March 14, 2000. A day later AOL yanked Gnutella, at the behest of Time Warner. Too late: 23 k users on Gnutella People had already downloaded and shared the program. Gnutella continues today, run by independent programmers. 29/85
Gnutella -- Justin Frankel and Tom Pepper 30/30
The ‘Animal’ GNU: Either of two large African antelopes (Connochaetes gnou or C. taurinus) having a drooping mane and beard, a long tufted tail, and curved horns in both sexes. Also called wildebeest. Gnutella = GNU: Recursive Acronym GNU’s Not Unix …. + GNU Nutella: a hazelnut chocolate spread produced by the Italian confectioner Ferrero …. 31/85
GNU l. GNU's Not Unix l 1983 Richard Stallman (MIT) established Free Software Foundation and Proposed GNU Project l. Free software is not freeware l. Free software is open source software l. GPL: GNU General Public License 32/85
About Gnutella No centralized directory servers n Pings the net to locate Gnutella friends n File requests are broadcasted to friends n ¨ Flooding, breadth-first search When provider located, file transferred via HTTP n History: n ¨ 3/14/00: release by AOL, almost immediately withdrawn 33/85
Peer-to-Peer Overlay Network Focus at the application layer 34/85
Peer-to-Peer Overlay Network End systems one hop (end-to-end comm. ) a TCP thru the Internet 35/85
Topology of a Gnutella Network 36/85
Gnutella: Issue a Request xyz. mp 3 ? 37/85
Gnutella: Flood the Request 38/85
Gnutella: Reply with the File Fully distributed storage and directory! xyz. mp 3 39/85
So Far n: number of participating nodes Centralized : - Directory size – O(n) - Number of hops – O(1) n Flooded queries: - Directory size – O(1) - Number of hops – O(n) n 40/85
We Want n Efficiency : O(log(n)) messages per lookup n Scalability : O(log(n)) state per node n Robustness : surviving massive failures 41/85
How Can It Be Done? n How do you search in O(log(n)) time? p. Binary search n You need an ordered array n How can you order nodes in a network and data objects? p. Hash function! 42/85
Example of Hasing Object ID (key): AABBCC Shark SHA-1 Object ID (key): DE 11 AC SHA-1 194. 90. 1. 5: 8080 43/85
Basic Idea Publish (H(y)) P 2 P Network Object “y” Objects have hash keys Join (H(x)) Peer “x” H(y) H(x) y Hash key Peer nodes also x have hash keys in the same hash space Place object to the peer with closest hash keys 44/85
Mapping Keys to Nodes 0 M - a node - an data object 45/85
Viewed as a Distributed Hash Table Hash table 0 2128 -1 Peer node Internet 46/85
DHT n n Distributed Hash Table Input: key (file name) Output: value (file location) Each node is responsible for a range of the hash table, according to the node’s hash key. Objects are placed in (managed by) the node with the closest key It must be adaptive to dynamic node joining and leaving 47/85
How to Find an Object? Hash table 0 2128 -1 Peer node 48/85
Simple Idea n Track peers which allow us to move quickly across the hash space ¨a peer p tracks those peers responsible for hash keys (p+2 i-1), i=1, . . , m Hash table 0 i i+22 i+24 i+28 2128 -1 Peer node 49/85
DHT example: Chord -- Ring Structure Circular 6 -bit ID space N 8 knows of only six other nodes. O(log n) states per node 50/85
DHT example: Chord -- Ring Structure O(log n)-hop query cost 51/85
Classification of P 2 P systems n Hybrid P 2 P – Preserves some of the traditional C/S architecture. A central server links between clients, stores indices tables, etc p Napster n Unstructured P 2 P – no control over topology and file placement p Gnutella, Morpheus, Kazaa, etc n Structured P 2 P – topology is tightly controlled and placement of files are not random p Chord, CAN, Pastry, Tornado, etc 52/85
What’s next in the future? n P 2 P NEVs (Networked Virtual Environments) ¨ P 2 P MMOGs (Massively Multiplayer Online Games) ¨ P 2 P 3 D Scene Streaming 53/85
P 2 P-NVE: Peer-to-Peer Networked Virtual Environment Part of the following slides are adapted from www. movesinstitute. org/~mcgredo/NVE. ppt
NVE Examples n Commercial: ¨ FPS: America’s Army ¨ MMOG Quake, Unreal, Ever. Quest, World of Warcraft (Wo. W), Lineage, Second Life, FPS (first person shooter) Research: NPSNET, Dive, MASSIVE n Military: Close Combat Tactical Trainer, SIMNET n
Americas Army
Massively Multiplayer Online Games n MMOGs are growing quickly ¨ 8 million registered users for World of Warcraft ¨ Over 100, 000 concurrent players ¨ Billion-dollar business Adaptive Computing and Networking Lab, CSIE, NCU 57
Adaptive Computing and Networking Lab, CSIE, NCU 58
Adaptive Computing and Networking Lab, CSIE, NCU 59
60
61
Adaptive Computing and Networking Lab, CSIE, NCU 62
Close Combat Tactical Trainer
NVE (1) Networked Virtual Environments (NVEs) are computer-generated virtual world where multiple geographically distributed users can assume virtual representatives (or avatars) to concurrently interact with each other n A. K. A. Distributed Virtual Environments (DVEs) n 64
NVE (3) n 3 D virtual world with ¨ People (avatar) ¨ Objects ¨ Terrain ¨ Agents ¨… n Each avatar can do a lot of operations ¨ Move ¨ Chat ¨ Other actions… 65
NVE Components • Graphic display n • User input and communication n • Processing/CPU n • Data network n
NVE Components: Graphics n n The display and CPU have become astonishingly cheap in the last few years We typically need to draw in 3 D on the display. The great thing about standards is that there are so many to choose from. Open. GL, X 3 D, Java 3 D Varying degrees of realism, with FPS emphasizing photorealism and others like Ever. Quest or Sims Online sacrificing graphics for better gameplay in other aspects
NVE User Interfaces n n You can have all sorts of input/output devices at the user’s location. With commercial games it is often a display, keyboard, and mouse. Military simulations may have more elaborate user environments, such as a mockup of an M 1 tank interior Some fancy UIs, such as head-mounted displays, caves, haptic feedback, data gloves, etc.
Processing/CPU Used for physics, AI, agent behavior, some graphics, networking n Still on Moore’s law curve. 4+ GHz machines are cheap and plentiful. n
Data Network n n n Networks are a real bottleneck in NVE design. Why? Players can send out position updates at 30/sec X 50 bytes/update + 42 bytes/packet overhead = 22, 000 bits/sec/player Even a 1 mbit/sec connection can run out of bandwidth after 40 -50 players 4/26/01
Data Network Latency is another big implementation problem. The position updates that arrive at hosts are always out of date, so we only know where the object was in the past n This may be a big problem (wide area network across a satellite link) or a small problem (everyone on a LAN) n
Issues for NVEs n Scalability ¨ To n Consistency ¨ All n accommodate as many as participants have the same view of object states Persistency ¨ All contents (object states) in NVE need to exist persistently n Reliability ¨ Need n to tolerate H. W and S. W. failures Security ¨ To prevent cheating and to keep user information and game state confidentially. 72
Architectures NVE architectures fall between two extreme poles: peer-to-peer and clientserver n In a Client-Server architecture, a server is responsible for sending out updates to the other hosts in the NVE n In a P 2 P architecture, all hosts communicate with other hosts n
Architectures (Client-Server) Popular with commercial game engines
Architectures (P 2 P) More popular in research and military
The Scalability Problem (1) n Client-server: has inherent resource limit Resource limit Adaptive Computing and Networking Lab, CSIE, NCU 76
The Scalability Problem (2) n Peer-to-Peer: Use the clients’ resources Resource limit Adaptive Computing and Networking Lab, CSIE, NCU 77
You only need to know some participants Area of Interest (AOI) ★: self ▲: neighbors Adaptive Computing and Networking Lab, CSIE, NCU 78
Voronoi-based Overlay Network : VON n n n Observation: ¨ for virtual environment applications, the contents we want are messages from AOI neighbors ¨ Content discovery is a neighbor discovery problem Solve the Neighbor Discovery Problem in a fullydistributed, message-efficient manner. Specific goals: ¨ Scalable Limit & minimize message traffics ¨ Responsive Direct connection with AOI neighbors 79
Voronoi Diagram n 2 D Plane partitioned into regions by sites, each region contains all the points closest to its site Neighbors Region Site 80
Design Concepts Use Voronoi to solve the neighbor discovery problem ¨ ¨ ¨ Each node constructs a Voronoi of its neighbors Identify enclosing and boundary neighbors Mutual collaboration in neighbor discovery ● node i and the big circle is its AOI ■ enclosing neighbors ▲ boundary neighbors ★ both enclosing and boundary neighbors ▼ normal AOI neighbors ◆ irrelevant nodes 81
Procedure (JOIN) 1)Joining node sends coordinates to any existing node Join request is forwarded to acceptor 2)Acceptor sends back its own neighbor list joining node connects with other nodes on the list Acceptor’s region Joining node 82
Procedure (MOVE) 1)Positions sent to all neighbors, mark messages to B. N. checks for overlaps between mover’s AOI and its E. N. 2)Connect to new nodes upon notification by B. N. Boundary neighbors New neighbors 83
Procedure (LEAVE) 1)Simply disconnect 2)Others then update their Voronoi new B. N. is discovered via existing B. N. Leaving node (also a B. N. ) New boundary neighbor 84
Q&A 85/85
