Google Map Reduce A DISTRIBUTED COMPUTING SOLUTION Outlines

Google Map. Reduce A DISTRIBUTED COMPUTING SOLUTION

Outlines �Introduction �Map. Reduce Model �Implementation �Refinements �Q&A

Introduction �Map. Reduce is designed to process large amount of distributed raw data. Machine learning, clustering data, query, graph-computation… The most significant use to date is indexing. �What’s the issues to be concerned?

Introduction �Abstraction �Reliability Fault-tolerance Load-balancing �Efficiency Parallelization Data distribution

Brief Ideas �User defines the input data into key/value pairs. �User defines Mapper function to process key/value pairs into intermediate key/value pairs. �User defines Reducer function to process intermediate key/value pairs and producing results.

Brief Ideas �Why it works? �An interface that abstracts the large-scale computation into two main function. �Automatic Parallelization �Robustness

Outlines �Introduction �Map. Reduce Model �Implementation �Refinements �Q&A

Model

Key/Value Pair �Input data type. �Consists of a key and a value. �Both key and value are of String type.

Map function �User specified function. �Processes some splits of input key/value pairs and produces intermediate key/value pairs (key, value)[] map((key, value)) �The output should be sorted (by key) and flushed to disk. Sorting makes sure pairs with the same key are grouped together.

Intermediate Key/Value Pairs �Output of mappers. �Intermediate key/value pairs will be further shuffled(grouped) by reducer. The process is implemented by user specified partitioning function. Ex. Hash(key) mod R Now the pairs become “key/value[]” pair

Reduce function �Another user specified function. �Integrate key and list of values into result output. value[] reduce((key, value[])) �Usually the result contains only one value or even none.

Example : Word Counter map( String key, String value ){ for each word w in value{ Emit. Intermediate(w, ” 1”); } } ……then all intermediates having the same w will be shuffled into (w, (” 1”, …, ” 1”)) Reduce( String key, Iterator values ){ int result = 0; for each v in values{ result += parse. Int(v); } Emit(to. String(result)); }

Other examples �Counting URL access frequency Map function processes URL log and output <URL, 1> Reduce function adds together all values for the same URL and emits the count for each URL. �Reverse Web-Link Graph Map function outputs <target, source> Reduce function outputs <target, list(source)> �Inverted Index Map function processes document into <word, document. ID> Reduce function sorts document. ID and emits <word, list(document. ID)>

Using Map. Reduce Model �User specifies the parallelization level. �User specifies map and reduce function �User specifies partitioning function

Outlines �Introduction �Map. Reduce Model �Implementation �Refinements �Q&A

Implementation �A widely used implementation at Google. �Large cluster of commodity PCs, connected by switched Ethernet. Machine : dual-processor x 86 processors running Linux Memory : 2 -4 GB per machine Networking : 100 MB~1 GB/second per machine � Maybe slower because of bisection bandwidth. Storage : Inexpensive IDE disks directly attached to machines File system : GFS (High availability and reliability)

Execution �User program calls Map. Reduce function. �The execution steps are briefly illustrated below.

Execution Flow

Execution The library in the user program splits input data file into M splits. Then it starts the program on a cluster of machines. 1. M is chosen so that each splits will be 16~64 MB per piece. 16 and 64 can be configured by user. � Locality optimization One of the machines will be elected as the master, and others as workers.

Execution 2. Master assumes there are M map tasks and R reduce tasks to be assigned. Then it assigns the idle workers a map task or reduce task. R also can be manually configured. But usually constrained because each reduce worker generate a separate file. Usually M and R are chosen to be a multiple of the number of worker machines. � � Better load balancing Speeds up worker-failing recovery.

Execution 3. The worker which is assigned the ith map task will read the ith input splits. It further parse the data and feeds the key/value pairs into map function, then buffered the output in the memory.

Execution 4. Periodically the memory will be flushed to local disk. Now the intermediate key/value pairs will be partitioned in R groups. By the partitioning function. (ex. Hash(key) mod R) Finishing the storing, the worker notice the master the locations where it stores the data. The data will be forwarded to reduce worker later.

Execution 5. The master notify a reduce worker the location of the locations of its input are. Then the ith reduce worker will reads the ith group of intermediate key/value pairs. Then sort the pairs read. Sorting ensures that the pairs with the same key will be put together. Sorting is needed because there maybe multiple keys to process in a given task. If the amount of the data exceeds memory, external sort will be used.

Execution 6. After sorting each unique key encountered, it collects all values follow the key and feeds the key/values pair to reduce function. The output will be appended to its final output file.

Execution 7. When all map tasks and reduce tasks completes, the master will wakeup the user program and the Map. Reduce function returns. There will be exactly R output files. Usually the output files are given by user program, so no return value is needed.

Illustration REDUCE 2. 4. 5. 1. Initializing 3. Read 6. 7. End Read Assign Store Output operation Inputs intermediate and tasks complete Done! 0 1 MAP 2 3 Reducers Mappers

Master �The scheduler of the whole Map. Reduce process. �It pings each worker periodically. �It keeps the state of each task (either map or reduce) in memory. State is of three possible value { idle, in-progress, completed } Workers completing their task will let the master know. So the master knows where to get the intermediate key/value pairs. �It keeps all intermediate pairs’ location and propagates them to reduce workers.

Fault Tolerance �What can go wrong? preemption 78. 3% exceeded resources 10. 8% crashed 9. 2% machine failure 1. 6% �Worker fails Offline, Straggler �Master fails �Input and output file consistency is ensured by file system (like GFS).

Worker Failure (Offline) �Master pings worker periodically to detect failing worker. �In-progress task assigned to the worker will be reassigned. �To map worker, any completed map task (their state should be completed) done by the worker will be set back to idle state and can be assigned to other worker again. Because of the loss of the output in the worker. All reduce workers will be notified for the re-execution.

Worker Failure (Straggler) �The worker that take unusual long time to complete the task. Bad disk , resource competition, ill caching mechanism… �The master assigns a backup task to the in-progress task. Either the primary or backup will complete the task first, and the master just ignores the later one. �A backup-disabled Map. Reduce takes about 44% longer times to accomplish the work.

Data Consistency �Every map worker generates R temporary files, and every reduce worker generate 1 output file. �All reduce worker writes to a temporary file. The file will be (atomically) renamed to the final output. The atomicity is guaranteed by underlying file system.

Master Failure �Master periodically checkpoints its data structure. �If master dies, a new copy can be started from the last checkpoints. �This happens relatively unlikely. (There will always be one master) �Real implementation just abort Map. Reduce operation and can be retry later.

Outlines �Introduction �Map. Reduce Model �Implementation �Refinements �Q&A

Locality �GFS generally has 3 replicas for each file chunk (on different machines). �Master will try to assign a map task on a machine that contains a replica of input. The locality will help saving network bandwidth. If impossible, than a nearby machine can be selected. Empirically a large fraction of input are read locally.

Partitioning Function �Can use simple hash-modular method. �Sometimes it will be better if we partition the data so that the output file will be well-organized. Ex: Hash(Hostname(key. URL)) mod R

Combiner �In some cases there will be much repetition of intermediate pairs. �An intermediate function between output of map function and intermediate key/value pairs. Ex. Thousands of <word, 1> can be merged to <word, k> Saving a lot of network bandwidth between reduce worker and map worker.

Skipping Bad Sectors �Sometimes there are bugs in user code so that worker fails to process some records. �If a records failed, the argument’s ID will be sent to the master. �If the ID is seen more than once, than the operation just skip that record. This service can be manually turned off.

Counter �Sometimes user will need to accumulate some number of the occur of special events. The words processed, the Chinese documents processed… �User can create a Counter object and increment it in map or reduce function. �The counter value is propagated to master on periodic ping. All values will later be aggregated. Only successful task’s value will be aggregated

Outlines �Introduction �Map. Reduce Model �Implementation �Refinements �Q&A

Q&A �Any questions?