Introduction Companion slides for Distributed Computing Through Combinatorial

Introduction Companion slides for Distributed Computing Through Combinatorial Topology Maurice Herlihy & Dmitry Kozlov & Sergio Rajsbaum Distributed Computing though Combinatorial Topology 1

In the Beginning … 1 0 1 0 a computer was just a Turing machine … Distributed Computing though Combinatorial Topology 2

Today ? ? ? Computing is co-ordination and communication Distributed Computing though Combinatorial Topology 3

Distributed computations unfold in time! No, distributed computations are static mathematical objects! Operational versus combinatorial approaches … Background picture: School of Athens, Rafael https: //www. flickr. com/photos/stefanorometours/6796909455/in/photolist-6 mya 9 T-bm. BWo 4 -bm. C 4 SR-bm. BXr. X-5 c https: //creativecommons. org/licenses/by/2. 0/legalcode 4

Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 5

Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 6

There are Many Models http: //sumptuoussynthphonys. blogspot. com/2013 Distributed Computing though Combinatorial Topology 7

There are Many Models Communication? http: //sumptuoussynthphonys. blogspot. com/2013 Distributed Computing though Combinatorial Topology 8

There are Many Models Communication? Failures? http: //sumptuoussynthphonys. blogspot. com/2013 Distributed Computing though Combinatorial Topology 9

There are Many Models Communication? Failures? http: //sumptuoussynthphonys. blogspot. com/2013 Timing? Distributed Computing though Combinatorial Topology 10

Communication http: //commons. wikimedia. org/wiki/File: Pennyblack-pd. jpg Message-Passing Distributed Computing though Combinatorial Topology 11

Communication http: //commons. wikimedia. org/wiki/File: Pennyblack-pd. jpg http: //commons. wikimedia. org/wiki/File: Rosetta. Stone. Detail. jpg Message-Passing Read-Write Memory Distributed Computing though Combinatorial Topology 12

Communication http: //commons. wikimedia. org/wiki/File: Pennyblack-pd. jpg http: //commons. wikimedia. org/wiki/File: Rosetta. Stone. Detail. jpg Message-Passing http: //pixabay. com/en/cube-black-block-box-3 d-250082/ Read-Write Memory Black-Box Memory 13 Distributed Computing though Combinatorial Topology

Message-Passing Prof. James Moriarty Brown University Providence RI 02912 Mr. S. Holmes 221 B Baker Street London NW 1 6 XE 14

Read-Write Memory http: //commons. wikimedia. org/wiki/File: Rosetta. Stone. Detail. jpg http: //www. alpa. ch/en/news/2011/dead-sea-scrolls Distributed Computing though Combinatorial Topology 15

Read-Write Models write & read individual locations write & take memory snapshot Group writes together then takes snapshots together 16

Layered Read-Write Memory 17

Black Box Memory Compare-and-swap Fetch-and-add 18

Failures Crash failures: processes halt How many? Which ones? Distributed Computing though Combinatorial Topology 19

Wait-Free Failure Model All but one may crash 20

t-Resilient Failure Model At most t may crash Here, t = 1. 21

Correlated Failures Processes on same server may crash 22

Adversaries http: //pixabay. com/en/chess-figures-game-play-strategy-145184/ Walt Disney Distributed Computing though Combinatorial Topology 23

Adversaries Determine which sets of processes can halt. http: //pixabay. com/en/chess-figures-game-play-strategy-145184/ Walt Disney Distributed Computing though Combinatorial Topology 24

Adversaries Determine which sets of processes can halt. “worst-case” scenario http: //pixabay. com/en/chess-figures-game-play-strategy-145184/ Walt Disney Distributed Computing though Combinatorial Topology 25

http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Distributed Computing though Combinatorial Topology 26

http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Distributed Computing though Combinatorial Topology 27

http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Synchronous Distributed Computing though Combinatorial Topology 28

http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Synchronous Processes do not share a clock Distributed Computing though Combinatorial Topology 29

http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Synchronous Processes do not share a clock Asynchronous Distributed Computing though Combinatorial Topology 30

http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Synchronous Processes do not share a clock Asynchronous Processes have approximatelysynchronized clocks Distributed Computing though Combinatorial Topology 31

http: //commons. wikimedia. org/wiki/File: Watch_automatic Timing Models Processes share a clock Synchronous Processes do not share a clock Asynchronous Processes have approximatelysynchronized clocks Semi-synchronous Distributed Computing though Combinatorial Topology 32

Synchronous 33

Synchronous Failures ! detection easy 34

Asynchronous 35

Asynchronous Failures ? ? detection impossible 36

Semi-Synchronous 37

Semi-Synchronous Failures ! detection slow 38

Computation Model Space asynchronous semi-synchronous adversaries t-resilient wait-free messages read-write black-box Which combinations make sense? 39

Multicores Asynchronous Wait-free Shared Memory Sun Microsystems http: //www. bit-tech. net/hardware/cpus/2011/01/03/intel-sandy-bridge-review/ 40

Distributed Computing Internet http: //commons. wikimedia. org/wiki/File: Internet_hosts. PNG Asynchronous Message-passing Sensor network http: //commons. wikimedia. org/wiki/File: XBee_Series_2_with_Whip_Antenna. jpg 41

Parallel Computing Synchronous Message-passing (or shared memory) http: //commons. wikimedia. org/wiki/File: Geforce 2_mx 400_gpu. JPG GPU 42

Local Views 110 Each process has a 3 -bit local view Distributed Computing though Combinatorial Topology 43

Multiple Local Views 110 111 local views differ by 1 bit Distributed Computing though Combinatorial Topology 44

Multiple Local Views 110 local views differ by 1 bit but no process knows which one Distributed Computing though Combinatorial Topology 111 45

Multiple Local Views 110 111 each view is represented by a labeled vertex Distributed Computing though Combinatorial Topology 46

Global States 110 111 compatible views represented by an edge Distributed Computing though Combinatorial Topology 47

000 100 001 101 010 110 011 111 All possible global states 110 111 Distributed Computing though Combinatorial Topology (where blue has 111)

Communication 110 111 Each process sends local view to the other Distributed Computing though Combinatorial Topology 49

Communication 110 111 Each process sends local view to the other but at most one message may be lost! Distributed Computing though Combinatorial Topology 50

One Communication Round 110 ? 111 110 111 Distributed Computing though Combinatorial Topology 111 ? 51

One Communication Round 110 ? 111 110 111 ? 1 Lost Distributed Computing though Combinatorial Topology 52

One Communication Round 110 ? 111 110 1 Lost 110 111 ? none Lost Distributed Computing though Combinatorial Topology 53

One Communication Round 110 ? 111 110 1 Lost 110 111 none Lost Distributed Computing though Combinatorial Topology 111 ? 1 Lost 54

One Communication Round 110 ? 111 110 111 Distributed Computing though Combinatorial Topology 111 ? 55

All possible global states after one round unreliable communication 110 ? 110 111 ? Distributed Computing though Combinatorial Topology

000 100 001 101 010 110 011 111

000 100 001 101 010 110 011 111 Informally …. Unreliable communication does not change “topology” of global states

Reliable Communication? 110 ? 111 110 111 Distributed Computing though Combinatorial Topology 111 ? 59

Reliable Communication? 111 110 111 Distributed Computing though Combinatorial Topology 60

000 100 001 101 010 110 011 111 Distributed Computing though Combinatorial Topology

Tasks Distributed Computing though Combinatorial Topology 62

19 32 Tasks Possible set of input values 21 Distributed Computing though Combinatorial Topology 63

19 32 Tasks Possible set of input values 21 Finite computation Distributed Computing though Combinatorial Topology 64

Tasks 19 32 Possible set of input values 21 Finite computation 19 19 Possible set of output values 32 32 19 21 21 32 21 Distributed Computing though Combinatorial Topology 65

Task Inputs Specification computation Outputs decision Protocol

Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 67

Muddy Children 11: 00 Distributed Computing though Combinatorial Topology 68

Muddy Children 11: 01 Distributed Computing though Combinatorial Topology 69

Muddy Children 12: 00 At least one of you is dirty! Distributed Computing though Combinatorial Topology 70

Muddy Children 12: 00 You may not communicate! Distributed Computing though Combinatorial Topology 71

Muddy Children 12: 00 When you realize you are dirty, confess on the hour! Distributed Computing though Combinatorial Topology 72

Muddy Children 1: 00 (silence …) Distributed Computing though Combinatorial Topology 73

Muddy Children 2: 00 Me! Distributed Computing though Combinatorial Topology 74

Operational Explanation 1: 00 Distributed Computing though Combinatorial Topology 75

Operational Explanation Others are clean, so I must be dirty. Distributed Computing though Combinatorial Topology 1: 00 76

Operational Explanation Others are clean, so I must be dirty. 1: 00 Me! Distributed Computing though Combinatorial Topology 77

Operational Explanation 1: 01 Distributed Computing though Combinatorial Topology 78

Operational Explanation He was quiet, so I must be dirty. 1: 01 He was quiet, so I must be dirty. Distributed Computing though Combinatorial Topology 79

Combinatorial Explanation 12: 00 Distributed Computing though Combinatorial Topology 80

Combinatorial Explanation 12: 00 Distributed Computing though Combinatorial Topology 81

Combinatorial Explanation 12: 00 Each process has its own input Distributed Computing though Combinatorial Topology 82

Combinatorial Explanation 01? ? 11 12: 00 Each process has its own input 0? 1 Distributed Computing though Combinatorial Topology 83

Combinatorial Explanation 01? ? 11 12: 00 Global State 0? 1 Distributed Computing though Combinatorial Topology 84

11: 59 all clean 00? 0? 0 ? 00 0? 1 01? ? 01 ? 1 0 10? 1? 0 ? 11 1? 1 11? all dirty Distributed Computing though Combinatorial Topology

12: 01 00? 0? 0 ? 00 0? 1 01? ? 01 ? 1 0 10? 1? 0 ? 11 1? 1 11? all dirty Distributed Computing though Combinatorial Topology

1: 01 0? 1 01? ? 01 ? 1 0 10? 1? 0 ? 11 1? 1 11? all dirty Distributed Computing though Combinatorial Topology

2: 01 ? 11 1? 1 11? all dirty Distributed Computing though Combinatorial Topology

Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 89

Coordinated Attack Alice Bob Red army wins If both sides attack together Distributed Computing though Combinatorial Topology 90

The Two Generals Alice Bob Red generals send messengers across the valley Distributed Computing though Combinatorial Topology 91

The Two Generals Alice Bob Messengers don’t always make it Distributed Computing though Combinatorial Topology 92

Your Mission Design a protocol to ensure that Alice and Bob attack simultaneously Distributed Computing though Combinatorial Topology 93

Theorem There is no protocol that ensures that the Red armies attack simultaneously Distributed Computing though Combinatorial Topology 94

Operational Proof Suppose Bob receives a message at 1: 00 saying “attack at Dawn”. Distributed Computing though Combinatorial Topology 95

Operational Proof Suppose Bob receives a message at 1: 00 saying “attack at Dawn”. Are we done? Distributed Computing though Combinatorial Topology 96

Operational Proof Suppose Bob receives a message at 1: 00 saying “attack at Dawn”. Are we done? No, because Alice doesn’t know if Bob got that message … Distributed Computing though Combinatorial Topology 97

Operational Proof So Bob sends an acknowledgment to Alice … Distributed Computing though Combinatorial Topology 98

Operational Proof So Bob sends an acknowledgment to Alice … Are we done? Distributed Computing though Combinatorial Topology 99

Operational Proof So Bob sends an acknowledgment to Alice … Are we done? No, because Bob doesn’t know if Alice got that message … Distributed Computing though Combinatorial Topology 100

Operational Proof So Bob sends an Thi s go acknowledgment to Alice no es on pro… toco forev l is er … Are we done? pos so sibl No, because Bob doesn’t e know if Alice got that message … Distributed Computing though Combinatorial Topology 101

Attack at dawn! Attack at noon! Noon Bob is Alice is Distributed Computing though Combinatorial Topology

delivered lost 1: 00 PM Distributed Computing though Combinatorial Topology delivered

delivered lost 2: 00 PM Distributed Computing though Combinatorial Topology delivered

delivered lost delivered protocol graph Attack at dawn! decision map Don’t attack! output graph Attack at noon!

delivered lost delivered protocol graph These edges go here dawn! noon! output graph

This graph is connected Distributed Computing though Combinatorial Topology

This graph is not connected Distributed Computing though Combinatorial Topology

delivered lost delivered Map not allowed to “tear” protocol complex

protocol graph decision map output graph

Operational Reasoning http: //commons. wikimedia. org/wiki/File: Professor_Lucifer_Butts. gif Distributed Computing though Combinatorial Topology 111

Combinatorial Reasoning http: //commons. wikimedia. org/wiki/File: Blake_ancient_of_days. jpg 112

Combinatorial Reasoning Model-independent properties … http: //commons. wikimedia. org/wiki/File: Blake_ancient_of_days. jpg 113

Combinatorial Reasoning Model-independent properties … … restricted model-dependent reasoning http: //commons. wikimedia. org/wiki/File: Blake_ancient_of_days. jpg 114

This work is licensed under a Creative Commons Attribution. Share. Alike 2. 5 License. • You are free: – to Share — to copy, distribute and transmit the work – to Remix — to adapt the work • Under the following conditions: – Attribution. You must attribute the work to “Distributed Computing Through Combinatorial Topology – ” (but not in any way that suggests that the authors endorse you or your use of the work). – Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. • For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to – http: //creativecommons. org/licenses/by-sa/3. 0/. • • Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights. Distributed Computing though Combinatorial Topology