2016 2017 http www cslab ece ntua grcoursesdistrib
Δίκτυα Ομότιμων Κόμβων Κατανεμημένα Συστήματα 2016 -2017 http: //www. cslab. ece. ntua. gr/courses/distrib 10/9/2021 Big Data related projects
Παράδειγμα: Διαμοιρασμός αρχείου • Γίνεται και με client-server F bits upload rate us d 4 Internet d 3 d 1 d 2 Download rates di 10/9/2021 Big Data related projects
Napster Store a directory, i. e. , filenames with peer pointers Filename Yesterday!. mp 3 Metadata Beatles, @ 147. 102. 5. 65: 1006 Penny. Lane. mp 3 Beatles, @ 128. 84. 92. 23: 1006 napster. com Servers S S Help!. mp 3 S Client machines (“Peers”) P P P 10/9/2021 Big Data related projects Beatles, @ 147. 102. 3. 10: 1006 P Store their own files
Napster 2. All servers search their lists napster. com Servers S Store peer pointers for all files S S Peers P P 1. Query P 3. Response P P P Store their own files 4. ping candidates 5. download from best host 10/9/2021 Big Data related projects
Gnutella Αποθήκευση αρχείων Servants (“Peers”) P P P Αλλά και κόμβωνγειτόνων P P Connected in an overlay graph (== each link is an implicit Internet path) 10/9/2021 Big Data related projects
Gnutella Query’s flooded out, ttl-restricted, forwarded only once P P TTL=2 P P P 10/9/2021 Big Data related projects
Gnutella Successful results Query. Hit’s routed on reverse path P P P P 10/9/2021 Big Data related projects
Kazaa 10/9/2021 Big Data related projects
Bit. Torrent Tracker Web Server . torre nt Web page with link to. torrent C A Peer [Leech] B Downloader Peer “US” [Leech] 10/9/2021 Big Data related projects [Seed]
Bit. Torrent Tracker Web Server Web page with link to. torrent n ou nn a t Ge ce C A Peer [Leech] B Downloader Peer “US” [Leech] 10/9/2021 Big Data related projects [Seed]
Bit. Torrent Tracker Web Server Web page with link to. torrent st li r e e e-p s on sp e R C A Peer [Leech] B Downloader Peer “US” [Leech] 10/9/2021 Big Data related projects [Seed]
Bit. Torrent Tracker Web Server Web page with link to. torrent Shake-hand C A Peer Sh [Leech] ak Peer e-h an d B Downloader Peer “US” [Leech] 10/9/2021 Big Data related projects [Seed]
Bit. Torrent Tracker Web Server Web page with link to. torrent pieces A Peer [Leech] pie ce s Peer B Downloader Peer “US” [Leech] 10/9/2021 Big Data related projects C [Seed]
Bit. Torrent Tracker Web Server Web page with link to. torrent pieces A Peer [Leech] pie ce ce s s Peer B Downloader Peer “US” [Leech] 10/9/2021 Big Data related projects C [Seed]
Bit. Torrent Tracker Web Server Web page with link to. torrent ce un o nn t-a e G st li r e e e-p s on sp e R A Peer [Leech] pieces pie ce ce s s Peer B Downloader Peer “US” [Leech] 10/9/2021 Big Data related projects C [Seed]
Παράδειγμα 10/9/2021 Big Data related projects
Παράδειγμα 10/9/2021 Big Data related projects
Τι θα θέλαμε; • Hash table: Δομή που συνδέει κλειδιά με τιμές • Κόστος αναζήτησης: Ο(1) Table Index • Ζεύγη (key, value) – (http: //www. cnn. com/foo. html, Web page) – (Help!. mp 3, 12. 78. 183. 2) 10/9/2021 Big Data related projects Values
DHT P P lookup(key) key value P P 10/9/2021 Big Data related projects P P
Παράδειγμα finger table • Ids 0 -120 -> 27 =128 > 120 οπότε 7 fingers • fn(i) = Successor(n+2 imod 120) Finger Table at N 80 N 114 i ft[i] 0 1 2 3 4 5 6 96 96 96 114 25 80 + 25 N 96 80 + 24 80 + 23 80 + 22 80 + 21 80 + 20 N 80 10/9/2021 Big Data related projects 80 + 26
Εύρεση key με finger table N 20 N 102 86 + 24 N 86 20 + 26 10/9/2021 Big Data related projects
Κίνητρο • Shopping cart service – 3 million checkouts κάθε μέρα – Εκατοντάδες χιλιάδες ταυτόχρονα sessions • Οι εφαρμογές πρέπει να μπορούν να παραμετροποιήσουν το Dynamo για επιθυμητό latency & throughput – Τουλάχιστον 99. 9% των read/write λειτουργιών σε λιγότερο από 1 sec – Ισορροπία ανάμεσα σε performance, cost, availability, durability guarantees. • Ανάγκη για availability (θυμήσου το θεώρημα CAP) – Eventual consistency – Partition tolerance – Availability (“always-on” experience) 10/9/2021 Big Data related projects
Παράδειγμα • 2 physical nodes, 5 virtual nodes 10/9/2021 Big Data related projects
Replication • Lazy replication (eventual consistency) – Ένα αίτημα put() επιστρέφει αμέσως (high write throughput) • Δεν περιμένει να διαδοθεί το update σε όλα τα replicas • Οδηγεί σε inconsistency, που λύνεται με object versioning 10/9/2021 Big Data related projects
Στατιστικά για οbject Versioning • • • Over a 24 -hour period 99. 94% of requests saw exactly one version 0. 00057% saw 2 versions 0. 00047% saw 3 versions 0. 00009% saw 4 versions Usually triggered by many concurrent requests issued busy robots, not human clients 10/9/2021 Big Data related projects
Quorums • Παράμετροι – N replicas – R readers – W writers • Στατικό quorum: R + W > N • Τυπικές τιμές για Dynamo: (N, R, W) == (3, 2, 2) 10/9/2021 Big Data related projects
Συγχρονισμός replicas • Σύγκριση 2 κόμβων που είναι συγχρονισμένοι – Two (key, value) pairs: (k 0, v 0) & (k 1, v 1) h 2 = hash(h 0 + h 1) h 0 = hash(v 0) h 1 = hash(v 1) Node 0 10/9/2021 Big Data related projects Equal h 2 = hash(h 0 + h 1) h 0 = hash(v 0) h 1 = hash(v 1) Node 1
Συγχρονισμός replicas • Σύγκριση 2 κόμβων που δεν είναι συγχρονισμένοι – One: (k 0, v 2) & (k 1, v 1) – The other: (k 0, v 0) & (k 1, v 1) h 4 = hash(h 2 + h 1) h 3 = hash(v 2) h 1 = hash(v 1) Node 0 10/9/2021 Big Data related projects Not equal h 2 = hash(h 0 + h 1) h 0 = hash(v 0) h 1 = hash(v 1) Node 1
- Slides: 80