Introduction Companion slides for Distributed Computing Through Combinatorial
Introduction Companion slides for Distributed Computing Through Combinatorial Topology Maurice Herlihy & Dmitry Kozlov & Sergio Rajsbaum Distributed Computing though Combinatorial Topology 1
In the Beginning … 1 0 1 0 a computer was just a Turing machine … Distributed Computing though Combinatorial Topology 2
Today ? ? ? Computing is co-ordination and communication Distributed Computing though Combinatorial Topology 3
Distributed computations unfold in time! No, distributed computations are static mathematical objects! Operational versus combinatorial approaches … Background picture: School of Athens, Rafael https: //www. flickr. com/photos/stefanorometours/6796909455/in/photolist-6 mya 9 T-bm. BWo 4 -bm. C 4 SR-bm. BXr. X-5 c https: //creativecommons. org/licenses/by/2. 0/legalcode 4
Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 5
Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 6
There are Many Models http: //sumptuoussynthphonys. blogspot. com/2013 Distributed Computing though Combinatorial Topology 7
There are Many Models Communication? http: //sumptuoussynthphonys. blogspot. com/2013 Distributed Computing though Combinatorial Topology 8
There are Many Models Communication? Failures? http: //sumptuoussynthphonys. blogspot. com/2013 Distributed Computing though Combinatorial Topology 9
There are Many Models Communication? Failures? http: //sumptuoussynthphonys. blogspot. com/2013 Timing? Distributed Computing though Combinatorial Topology 10
Communication http: //commons. wikimedia. org/wiki/File: Pennyblack-pd. jpg Message-Passing Distributed Computing though Combinatorial Topology 11
Communication http: //commons. wikimedia. org/wiki/File: Pennyblack-pd. jpg http: //commons. wikimedia. org/wiki/File: Rosetta. Stone. Detail. jpg Message-Passing Read-Write Memory Distributed Computing though Combinatorial Topology 12
Communication http: //commons. wikimedia. org/wiki/File: Pennyblack-pd. jpg http: //commons. wikimedia. org/wiki/File: Rosetta. Stone. Detail. jpg Message-Passing http: //pixabay. com/en/cube-black-block-box-3 d-250082/ Read-Write Memory Black-Box Memory 13 Distributed Computing though Combinatorial Topology
Message-Passing Prof. James Moriarty Brown University Providence RI 02912 Mr. S. Holmes 221 B Baker Street London NW 1 6 XE 14
Read-Write Memory http: //commons. wikimedia. org/wiki/File: Rosetta. Stone. Detail. jpg http: //www. alpa. ch/en/news/2011/dead-sea-scrolls Distributed Computing though Combinatorial Topology 15
Read-Write Models write & read individual locations write & take memory snapshot Group writes together then takes snapshots together 16
Layered Read-Write Memory 17
Black Box Memory Compare-and-swap Fetch-and-add 18
Failures Crash failures: processes halt How many? Which ones? Distributed Computing though Combinatorial Topology 19
Wait-Free Failure Model All but one may crash 20
t-Resilient Failure Model At most t may crash Here, t = 1. 21
Correlated Failures Processes on same server may crash 22
Adversaries http: //pixabay. com/en/chess-figures-game-play-strategy-145184/ Walt Disney Distributed Computing though Combinatorial Topology 23
Adversaries Determine which sets of processes can halt. http: //pixabay. com/en/chess-figures-game-play-strategy-145184/ Walt Disney Distributed Computing though Combinatorial Topology 24
Adversaries Determine which sets of processes can halt. “worst-case” scenario http: //pixabay. com/en/chess-figures-game-play-strategy-145184/ Walt Disney Distributed Computing though Combinatorial Topology 25
http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Distributed Computing though Combinatorial Topology 26
http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Distributed Computing though Combinatorial Topology 27
http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Synchronous Distributed Computing though Combinatorial Topology 28
http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Synchronous Processes do not share a clock Distributed Computing though Combinatorial Topology 29
http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Synchronous Processes do not share a clock Asynchronous Distributed Computing though Combinatorial Topology 30
http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Synchronous Processes do not share a clock Asynchronous Processes have approximatelysynchronized clocks Distributed Computing though Combinatorial Topology 31
http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Synchronous Processes do not share a clock Asynchronous Processes have approximatelysynchronized clocks Semi-synchronous Distributed Computing though Combinatorial Topology 32
Synchronous 33
Synchronous Failures ! detection easy 34
Asynchronous 35
Asynchronous Failures ? ? detection impossible 36
Semi-Synchronous 37
Semi-Synchronous Failures ! detection slow 38
Computation Model Space asynchronous semi-synchronous adversaries t-resilient wait-free messages read-write black-box Which combinations make sense? 39
Multicores Asynchronous Wait-free Shared Memory Sun Microsystems http: //www. bit-tech. net/hardware/cpus/2011/01/03/intel-sandy-bridge-review/ 40
Distributed Computing Internet http: //commons. wikimedia. org/wiki/File: Internet_hosts. PNG Asynchronous Message-passing Sensor network http: //commons. wikimedia. org/wiki/File: XBee_Series_2_with_Whip_Antenna. jpg 41
Parallel Computing Synchronous Message-passing (or shared memory) http: //commons. wikimedia. org/wiki/File: Geforce 2_mx 400_gpu. JPG GPU 42
Local Views 110 Each process has a 3 -bit local view Distributed Computing though Combinatorial Topology 43
Multiple Local Views 110 111 local views differ by 1 bit Distributed Computing though Combinatorial Topology 44
Multiple Local Views 110 local views differ by 1 bit but no process knows which one Distributed Computing though Combinatorial Topology 111 45
Multiple Local Views 110 111 each view is represented by a labeled vertex Distributed Computing though Combinatorial Topology 46
Global States 110 111 compatible views represented by an edge Distributed Computing though Combinatorial Topology 47
000 100 001 101 010 110 011 111 All possible global states 110 111 Distributed Computing though Combinatorial Topology (where blue has 111)
Communication 110 111 Each process sends local view to the other Distributed Computing though Combinatorial Topology 49
Communication 110 111 Each process sends local view to the other but at most one message may be lost! Distributed Computing though Combinatorial Topology 50
One Communication Round 110 ? 111 110 111 Distributed Computing though Combinatorial Topology 111 ? 51
One Communication Round 110 ? 111 110 111 ? 1 Lost Distributed Computing though Combinatorial Topology 52
One Communication Round 110 ? 111 110 1 Lost 110 111 ? none Lost Distributed Computing though Combinatorial Topology 53
One Communication Round 110 ? 111 110 1 Lost 110 111 none Lost Distributed Computing though Combinatorial Topology 111 ? 1 Lost 54
One Communication Round 110 ? 111 110 111 Distributed Computing though Combinatorial Topology 111 ? 55
All possible global states after one round unreliable communication 110 ? 110 111 ? Distributed Computing though Combinatorial Topology
000 100 001 101 010 110 011 111
000 100 001 101 010 110 011 111 Informally …. Unreliable communication does not change “topology” of global states
Reliable Communication? 110 ? 111 110 111 Distributed Computing though Combinatorial Topology 111 ? 59
Reliable Communication? 111 110 111 Distributed Computing though Combinatorial Topology 60
000 100 001 101 010 110 011 111 Distributed Computing though Combinatorial Topology
Tasks Distributed Computing though Combinatorial Topology 62
19 32 Tasks Possible set of input values 21 Distributed Computing though Combinatorial Topology 63
19 32 Tasks Possible set of input values 21 Finite computation Distributed Computing though Combinatorial Topology 64
Tasks 19 32 Possible set of input values 21 Finite computation 19 19 Possible set of output values 32 32 19 21 21 32 21 Distributed Computing though Combinatorial Topology 65
Task Inputs Specification computation Outputs decision Protocol
Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 67
Muddy Children 11: 00 Distributed Computing though Combinatorial Topology 68
Muddy Children 11: 01 Distributed Computing though Combinatorial Topology 69
Muddy Children 12: 00 At least one of you is dirty! Distributed Computing though Combinatorial Topology 70
Muddy Children 12: 00 You may not communicate! Distributed Computing though Combinatorial Topology 71
Muddy Children 12: 00 When you realize you are dirty, confess on the hour! Distributed Computing though Combinatorial Topology 72
Muddy Children 1: 00 (silence …) Distributed Computing though Combinatorial Topology 73
Muddy Children 2: 00 Me! Distributed Computing though Combinatorial Topology 74
Operational Explanation 1: 00 Distributed Computing though Combinatorial Topology 75
Operational Explanation Others are clean, so I must be dirty. Distributed Computing though Combinatorial Topology 1: 00 76
Operational Explanation Others are clean, so I must be dirty. 1: 00 Me! Distributed Computing though Combinatorial Topology 77
Operational Explanation 1: 01 Distributed Computing though Combinatorial Topology 78
Operational Explanation He was quiet, so I must be dirty. 1: 01 He was quiet, so I must be dirty. Distributed Computing though Combinatorial Topology 79
Combinatorial Explanation 12: 00 Distributed Computing though Combinatorial Topology 80
Combinatorial Explanation 12: 00 Distributed Computing though Combinatorial Topology 81
Combinatorial Explanation 12: 00 Each process has its own input Distributed Computing though Combinatorial Topology 82
Combinatorial Explanation 01? ? 11 12: 00 Each process has its own input 0? 1 Distributed Computing though Combinatorial Topology 83
Combinatorial Explanation 01? ? 11 12: 00 Global State 0? 1 Distributed Computing though Combinatorial Topology 84
11: 59 all clean 00? 0? 0 ? 00 0? 1 01? ? 01 ? 1 0 10? 1? 0 ? 11 1? 1 11? all dirty Distributed Computing though Combinatorial Topology
12: 01 00? 0? 0 ? 00 0? 1 01? ? 01 ? 1 0 10? 1? 0 ? 11 1? 1 11? all dirty Distributed Computing though Combinatorial Topology
1: 01 0? 1 01? ? 01 ? 1 0 10? 1? 0 ? 11 1? 1 11? all dirty Distributed Computing though Combinatorial Topology
2: 01 ? 11 1? 1 11? all dirty Distributed Computing though Combinatorial Topology
Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 89
Coordinated Attack Alice Bob Red army wins If both sides attack together Distributed Computing though Combinatorial Topology 90
The Two Generals Alice Bob Red generals send messengers across the valley Distributed Computing though Combinatorial Topology 91
The Two Generals Alice Bob Messengers don’t always make it Distributed Computing though Combinatorial Topology 92
Your Mission Design a protocol to ensure that Alice and Bob attack simultaneously Distributed Computing though Combinatorial Topology 93
Theorem There is no protocol that ensures that the Red armies attack simultaneously Distributed Computing though Combinatorial Topology 94
Operational Proof Suppose Bob receives a message at 1: 00 saying “attack at Dawn”. Distributed Computing though Combinatorial Topology 95
Operational Proof Suppose Bob receives a message at 1: 00 saying “attack at Dawn”. Are we done? Distributed Computing though Combinatorial Topology 96
Operational Proof Suppose Bob receives a message at 1: 00 saying “attack at Dawn”. Are we done? No, because Alice doesn’t know if Bob got that message … Distributed Computing though Combinatorial Topology 97
Operational Proof So Bob sends an acknowledgment to Alice … Distributed Computing though Combinatorial Topology 98
Operational Proof So Bob sends an acknowledgment to Alice … Are we done? Distributed Computing though Combinatorial Topology 99
Operational Proof So Bob sends an acknowledgment to Alice … Are we done? No, because Bob doesn’t know if Alice got that message … Distributed Computing though Combinatorial Topology 100
Operational Proof So Bob sends an Thi s go acknowledgment to Alice no es on pro… toco forev l is er … Are we done? pos so sibl No, because Bob doesn’t e know if Alice got that message … Distributed Computing though Combinatorial Topology 101
Attack at dawn! Attack at noon! Noon Bob is Alice is Distributed Computing though Combinatorial Topology
delivered lost 1: 00 PM Distributed Computing though Combinatorial Topology delivered
delivered lost 2: 00 PM Distributed Computing though Combinatorial Topology delivered
delivered lost delivered protocol graph Attack at dawn! decision map Don’t attack! output graph Attack at noon!
delivered lost delivered protocol graph These edges go here dawn! noon! output graph
This graph is connected Distributed Computing though Combinatorial Topology
This graph is not connected Distributed Computing though Combinatorial Topology
delivered lost delivered Map not allowed to “tear” protocol complex
protocol graph decision map output graph
Operational Reasoning http: //commons. wikimedia. org/wiki/File: Professor_Lucifer_Butts. gif Distributed Computing though Combinatorial Topology 111
Combinatorial Reasoning http: //commons. wikimedia. org/wiki/File: Blake_ancient_of_days. jpg 112
Combinatorial Reasoning Model-independent properties … http: //commons. wikimedia. org/wiki/File: Blake_ancient_of_days. jpg 113
Combinatorial Reasoning Model-independent properties … … restricted model-dependent reasoning http: //commons. wikimedia. org/wiki/File: Blake_ancient_of_days. jpg 114
This work is licensed under a Creative Commons Attribution. Share. Alike 2. 5 License. • You are free: – to Share — to copy, distribute and transmit the work – to Remix — to adapt the work • Under the following conditions: – Attribution. You must attribute the work to “Distributed Computing Through Combinatorial Topology – ” (but not in any way that suggests that the authors endorse you or your use of the work). – Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. • For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to – http: //creativecommons. org/licenses/by-sa/3. 0/. • • Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights. Distributed Computing though Combinatorial Topology
- Slides: 115