Seminar Information Management in the Web Gnutella Freenet

Seminar: Information Management in the Web Gnutella, Freenet and more: an overview of file sharing architectures Thomas Zahn 1

Peer-to-Peer - Introduction • "opposite" of Client/Server • no central servers information highly distributed • every peer acts as a client AND server • -> can query, reply to queries and route messages at the same time • every peer can directly "talk" to any other peer 2

Popular Peer-to-Peer Networks • • • Napster Gnutella Freenet Fast. Track (Kazaa) CHORD, CAN, PASTRY, TAPESTRY 3

Napster • • was used primarily for file sharing NOT a pure peer-to-peer network => hybrid system peer turns to central DB for querying (client/server) • peer downloads directly from other peer(s) (peer-to-peer) 4

Napster 5 6 4 central DB 2. Response 1 3. Download Request 1. Query 3 4. File 2 Peer 5

Gnutella - overview • • • pure peer-to-peer used for file sharing very popular => practically proven ? very simple protocol no routing "intelligence" messages are always broadcast 6

Gnutella - PING/PONG 3 6 Ping 1 Pong 3 Pong 6, 7, 8 Pong 6 Ping 1 1 Known Hosts: 2 3, 4, 5 Pong 5 2 Ping 1 5 Pong 7 Ping 1 Pong 2 Ping 1 Pong 4 6, 7, 8 4 7 Pong 8 8 Query/Response analogous 7

Gnutella - Pro & Con • VERY simple protocol => easy to implement • very little overhead • practically proven functionality (? ) • message broadcasts flood network • =>heavy network traffic • => bad, bad scalibility 8

Gnutella – Reachable Peers T=1 T=2 T=3 T=4 T=5 T=6 T=7 T=8 N=2 2 4 6 8 10 12 14 16 N=3 3 9 21 45 93 189 381 765 N=4 4 16 52 160 484 1, 456 4, 372 13, 120 N=5 5 25 105 425 1, 705 6, 825 27, 305 109, 225 N=6 6 36 186 936 4, 686 23, 436 117, 186 585, 936 N=7 7 49 301 1, 813 10, 885 65, 317 391, 909 2, 351, 461 N=8 8 64 456 3, 200 22, 408 156, 864 1, 098, 056 7, 686, 400 9

Gnutella – Generated Traffic in Bytes (1) T=1 T=2 T=3 T=4 T=5 T=6 T=7 T=8 N=2 166 332 498 664 830 996 1, 162 1, 328 N=3 249 747 1, 743 3, 735 7, 719 15, 687 31, 623 63, 495 N=4 332 1, 328 4, 316 13, 28 40, 172 120, 848 362, 876 1, 088, 960 N=5 415 2, 075 8, 715 35, 275 141, 515 566, 475 2, 266, 315 9, 065, 675 N=6 498 2, 988 15, 438 77, 688 388, 938 1, 945, 188 9, 726, 438 48, 632, 688 N=7 581 4, 067 24, 983 150, 479 903, 455 5, 421, 311 32, 528, 447 195, 171, 263 N=8 664 5, 312 37, 848 265, 600 1, 859, 864 13, 019, 712 91, 138, 648 637, 971, 200 • query message length: 83 bytes • simple query relaying (no responses) 10

Gnutella – Generated Traffic in Bytes (2) N=3 N=4 N=5 N=6 N=7 N=8 T=1 T=2 T=3 T=4 T=5 T=6 T=7 T=8 283. 68 1, 418. 4 4, 822. 56 13, 900. 3 36, 594. 7 91, 061. 3 218, 15 508. 638 378. 24 2, 647. 68 12, 860. 2 53, 710. 1 206, 897 758, 371 2, 688, 530 9, 306, 220 472. 8 4, 255. 2 26, 949. 6 147, 986 753, 17 3, 658, 050 17, 214, 200 79, 185, 000 567. 36 6, 240. 96 48, 793 332, 473 2, 105, 470 12, 743, 500 74, 798, 500 429, 398, 000 661. 92 8, 604. 96 80, 092. 3 651, 991 4, 941, 123 35, 823, 800 252, 000 1, 734, 360, 000 756. 48 11, 347. 2 122, 55 1, 160, 440 10, 242, 000 86, 526, 900 709, 521, 000 5, 693, 470, 000 • Mean percentage of users who typically share content: 30% • Mean perctg. of users who typically have responses to search queries: 40% • Mean number of search responses the typical respondent offers: 10 • Mean length of search responses the typical respondent offers: 60 "Standard client settings yield a whopping 17 MB generated in response to […] search query " 11

Freenet - Concepts • peer-to-peer file storage & retrieval system • every document has a globally unique ID • efficient (? ) retrieval algorithm – documents are retrieved with sublinear effort • routing based on likelihood of answer capability • focus on security 12

Freenet – Query Routing (1) • every peer maintains routing table • table contains known peers along with the IDs of the documents their are storing • a request is routed to the peer most likely to have an answer (closest matching ID) • responses are sent back upstream • intermediate peers also store document and augment their routing tables 13

Freenet – Query Routing (2) Routing Table B: 14, 20 17, 20 X: 47, 60 Doc Cache 1. Query for doc 5, 17, 89 89 17 Routing Table C: 19, 30 D: 17, 45, 51 51 Doc Cache 14, 20 14, 17, 20 A 2. Forward to best match C Routing Table B: 14, 20 Doc Cache 19, 30 3. C has no match -> backtrack B 4. Forward query to 2 nd best match 6. Route back response 5. Send back doc 17 D Routing Table B: 14, 20 Z: 105, 110 Doc Cache 17, 45, 51, 102, 205 14

Freenet – Document Insert • analogous to query routing • insert is routed to the peer most likely to be interested in new doc (closest matching ID) • intermediate peers cache document and augment routing tables • until TTL is reached 15

Freenet - Discussion • efficient routing algorithm (compared to Gnutella) • adequate security features/heuristics (the more popular a document, the more frequently it gets cached) • no metasearch • no updates, deletes possible • worst case query routing = DFS 16

FUtella – Concepts • peer-to-peer platform for general knowledge sharing • tries to model learning style of humans • content-based routing • combines and extends approaches from: – Gnutella (message format) – JXTA (peer groups) – JXTA Search (queryspaces and registrations) – Free. Net (routing of registration discoveries) 17

FUtella - Knowledge Groups Knowledge Group: Queryspace "Computer Architecture" Group Head: Peer E FUtella Net Inserts Registration E M 1 . . . Mi Members M 1 - Mi 18

FUtella - Knowledge Group Discovery 1 1. Discovery request "computer architecture" A Routing Table "computer" -> B "data base" -> X Registration Cache "computer": B "data base": X Routing Table "computer analysis" -> C "computer systems" -> D "data base" -> A Registration Cache "computer analysis" : Y "computer systems": Z "data base" : X C 2. Forward discovery request B Routing Table "computer" -> B "computer analysis" -> Y Registration Cache "computer": B "computer analysis": Y 3. C has no cached registration for "computer architecture -> backtrack Routing Table "computer" -> B "computer systems" -> Z "computer architecture" -> E Registration Cache "computer systems": Z "computer": B "computer architecture": E D 4. Forward discovery request to 2 nd best match 19

FUtella - Knowledge Group Discovery 2 A 6. Forward discovery response Routing Table "computer" -> B "computer architecture" -> D "data base" -> X Registration Cache "computer": B "computer architecture": E "data base": X B 5. Discovery response Containing registration "computer architecture": E Routing Table "computer analysis" -> C "computer architecture" -> D "computer systems" -> D "data base" -> A Registration Cache "computer analysis" : Y "computer architecture": E "computer systems": Z "data base" : X D Routing Table "computer" -> B "computer systems" -> Z "computer architecture" -> E Registration Cache "computer systems": Z "computer": B "computer architecture": E 20

Futella - Query Processing 1. Discovery request "computer architecture" 2. Forward discovery request A B 4. Forward discovery request to 2 nd best match 6. Forward discovery response C 3. C has no cached registration for "computer architecture -> backtrack D 7. Send query 8. Forward query to member E 9. Query response M 1 8. Forward query to member. . . 9. Query response 5. Discovery response containing cached registration Knowledge group "computer architecture" Mi 21

Futella - Test Results (1) Total Number of Messages 250000 # msg 200000 threshold 2 no threshold Gnutella 150000 100000 50000 0 static peers semi-dynamic peers 22

FUtella - Test Results (2) 23

Conclusion • first and second generation P 2 P systems still most widely used • practically proven • very flexible in terms of topology • bad scalibility (Gnutella) • no guaranteed lower bound on query effort (Freenet) • (scientificly) far better approach: DHTs (see next presentation) 24

Questions ? ? 25