Scalable PeertoPeer Networked Virtual Environment Master Thesis Oral
Scalable Peer-to-Peer Networked Virtual Environment Master Thesis Oral Examination Dept. of CSIE, Tamkang Univ. Advisor: Dr. Chen Jui-Fa Shun-Yun Hu 2005/01/07 1
Outline l l l Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion 2
What is Networked Virtual Environment (NVE)? l Virtual Reality + Internet l 3 D environment with people (avatar), objects, terrain, agents l Military simulations (’ 80) Massively Multiplayer Online Games (mid-‘ 90) l Trends: larger scale, more realistic simulation 3
4
NVE: A Shared Space 5
Issues for Creating NVE l l l Consistency (events/states) Responsiveness Security multiplayer Scalability Persistency Reliability (Fault-tolerance) massively multiplayer 6
The Scalability Problem l l l Many nodes on a 2 D plane ( > 1, 000) Message exchange with those within Area of Interest (AOI) How does each node receive the relevant messages? Area of Interest 7
A simple solution (point-to-point) Source: [Funkhouser 95] N * (N-1) connections ≈ O(N 2) Not scalable! 8
A better solution (client-server) Source: [Funkhouser 95] Message filtering at server to reduce traffic N connections = O(N) server is bottleneck 9
Current solution (server-cluster) Source: [Funkhouser 95] Still limited by servers. Expansive to deploy & maintain. 10
Scalability Analysis l Scalability constrains l l Computing resource Network resource Non-scalable system (CPU) (Bandwidth) vs. Scalable system Resource limit x: number of entities y: resource consumption at the limiting system component 11
What Next? l Strategies l l l Increase resource Decrease consumption Architectures l l Point-to-point (LAN) Client-server Server-cluster ? Peer-to-Peer More servers Message filtering Scale tens hundreds thousands millions 10^1 10^2 10^3 10^6 … 12
What is Peer-to-Peer (P 2 P)? [Stoica et al. 2003] l Distributed systems without any centralized control or hierarchical organization l Runs software with equivalent functionality l Examples l l l File-sharing: Napster, Gnutella, e. Donkey Distributed computing: SETI@Home (UC Berkeley) Vo. IP: Skype 13
Peer-to-Peer Overlay A P 2 P overlay network 2003] source: [Keller & Simon 14
Promise & Challenge of P 2 P l Promises l l l Growing resource, decentralized Scalable Commodity hardware Affordable Challenges l l Topology maintenance dynamic join/leave Efficient content retrieval no global knowledge 15
Issues for Creating P 2 P NVE Consistency (events/states) Responsiveness Security multiplayer massively multiplayer l Scalability Persistency Reliability (Fault-tolerance) l Consistency (topology) l l l P 2 P NVE 16
Related Works (1): Sim. MUD [Knutsson et al. 2004] (Univ. of Pennsylvania) l l l Pastry + Scribe Regions Coordinators (super-nodes) Fixed-size region Relay overhead 17
Related Works (2) [Kawahara et al. 2004] (Univ. of Tokyo) l l l Fully-distributed Nearest-neighbors List exchange High transmission Overlay partition 18
Related Works (3): Solipsis [Keller & Simon 2003] (France Telecomm R&D) l l l Links with AOI neighbor Mutual cooperation Inside convex hull Potentially slow discovery Inconsistent topology 19
Outline l l l Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion 20
Design Goals l Observation: l l for virtual environment applications, the contents we want are messages from AOI neighbors Content discovery is a neighbor discovery problem l Solve the Neighbor Discovery Problem in a fullydistributed, message-efficient manner. l Specific goals: l l Scalable Responsive Limit & minimize message traffics Direct connection with AOI neighbors 21
Voronoi Diagram l l 2 D Plane partitioned into regions by sites, each region contains all the points closest to its site Can be used to find k-nearest neighbor easily Neighbors Region Site 22
Design Concepts Use Voronoi to solve the neighbor discovery problem l l Identify enclosing and boundary neighbors Each node constructs a Voronoi of its neighbors Enclosing neighbors are minimally maintained Mutual collaboration in neighbor discovery Circle Area of Interest (AOI) White self Yellow enclosing neighbor (E. N. ) L. Blue boundary neighbor (B. N. ) Pink E. N. & B. N. Green AOI neighbor D. Blue unknown neighbor 23
Procedure (JOIN) 1) Joining node sends coordinates to any existing node Join request is forwarded to acceptor 2) Acceptor sends back its own neighbor list joining node connects with other nodes on the list Joining node Acceptor’s region 24
Procedure (MOVE) 1) Positions sent to all neighbors, mark messages to B. N. checks for overlaps between mover’s AOI and its E. N. 2) Connect to new nodes upon notification by B. N. Disconnect any non-overlapped neighbor Boundary neighbors Non-overlapped neighbors New neighbors 25
Procedure (LEAVE) 1) Simply disconnect 2) Others then update their Voronoi new B. N. is discovered via existing B. N. Leaving node (also a B. N. ) New boundary neighbor 26
Dynamic AOI Crowding within AOI can overload a particular node It’s better if AOI-radius can be adjusted in real time 27
Adjustment Conditions l AOI-radius decrease l l AOI-radius increase l l l Number of connections > maximum allowable connections Maximum connections not exceeded Current AOI-radius < preferred AOI-radius Delay counter l To avoid fluctuations 28
Demonstration Simulation video l l l General movements (20 nodes, 800 x 600 world) Local vs. global view Dynamic AOI adjustment 29
30
Outline l l l Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion 31
Simulation Method l C++ implementation of Voronoi-based algorithm l World size: 1000 x 1000, AOI: 150 Trials from 10 – 250 nodes Connection limit per node: 10 1000 time-steps l l l (~ 100 simulated seconds, assuming 10 updates/seconds) l Behavior model l Random movement: Constant velocity: Movement duration: random direction 5 units/step random (1 – 25 steps) 32
Consistency Metrics l Topology Consistency [Kawahara, 2004] Number of observed AOI neighbors Number of actual AOI neighbors l Drift Distance [Diot, 1999] Distance between observed position and actual position (average over all nodes) 33
Basic Model Topology Consistency 34
Basic Model Scalability (1) 35
Basic Model Scalability (2) 36
Dynamic AOI Model 37
Dynamic AOI Scalability (1) 38
Dynamic AOI Scalability (2) 39
Dynamic AOI Scalability (3) 40
Dynamic AOI Topology Consistency (1) 41
Dynamic AOI Topology Consistency (2) 42
Dynamic AOI Reliability (1) 43
Dynamic AOI Reliability (2) 44
Outline l l l Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion 45
Analysis of Design Consistency (Topology) l Topology is fully connected & consistent enclosing neighbors Responsiveness l Lowest latency direct connection, no relay Scalability l Resource-growing & decentralized resource consumption Reliability l Self-organizing for small number of node failures 46
P 2 P NVE Comparisons Sim. MUD Consistency (topology) Neighbor-list Solipsis exchange Supernode Neighbor listexchange (partitioning) Responsive- High ness overhead High overhead VON Neighbor notify&query notify (undiscovery (consistent) ) Medium overhead Low overhead Scalability Relied on Fullysupernode distributed Fullydistributed Reliability Long uptime N/A Selforganizing N/A 47
Problems of Voronoi Approach l Message traffic l l Circular round-up of nodes Redundant message sending (inherent to fully-distributed design) l Incomplete neighbor discovery l l Can happen with inconsistent / incorrect neighbor list Fast moving node 48
Outline l l l Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion 49
Conclusion l NVE scalability is achievable with P 2 P architecture and is a neighbor discovery problem l A promising solution: Voronoi-based P 2 P Overlay l l Leverage knowledge of each peer to maintain topology Properties l l l Scalable: fully-distributed, dynamic AOI Efficient: low irrelevant messages, zero relay Robust: consistent and self-organizing topology 50
Potential Applications l Online games Relieve server from position updates in current MMOGs l Military Enable large-scale, affordable military training simulation l 3 D Web Provide multi-user interactivity to static 3 D world l Scientific simulations Distribute spatial simulation requiring frequent synchronization 51
Future Works l Short-term l l Reliability measurements latency, packet loss, node fail Distributed event/state consistency Recovery from overlay partition Long-term l l l Persistency issue (P 2 P-based database) Security issue (protection from malicious nodes) 3 D content distribution (3 D streaming on P 2 P) Massive, persistent 3 D environment sharable by all! 52
Acknowledgements l l l Dr. Jui-Fa Chen (陳瑞發老師) Dr. Wei-Chuan Lin (林偉川老師) Members of the Alpha Lab, TKU CS Guan-Ming Liao (廖冠名) Dr. Chin-Kun Hu (胡進錕老師) LSCP, Institute of Physics, Academia Sinica Joaquin Keller Bart Whitebook Jon Watte (France Telecomm R&D, Solipsis) (butterfly. net) (there. com) Dr. Wen-Bing Horng Dr. Jiung-yao Huang (洪文斌老師) (黃俊堯老師) 53
Inconsistency caused by d. AOI 54
Reliability (0 -500 steps) 55
Reliability (501 -1000 steps) 56
- Slides: 56