SESSION CODE DEV 314 Huseyin YILDIZ Software Design

SESSION CODE: DEV 314 Huseyin YILDIZ Software Design Engineer Microsoft Corporation

CPU 1 CPU 2 CPU 3 CPU 4 Op 1 Op 2 Op 3 Op 4 Op 5

IEnumerable<Record> records = Get. Records(); List<Result> results = new List<Result>(); foreach(Record record in records) { Result res = Process. Record(record); // non-trivial computation per record if (Filter(res)) results. Add(res); // check if result matches our criteria, if so add it into results } // can also be expressed as a simple LINQ statement List<Result> results = records. Select(rec => Process. Record(rec)). Where(res => Filter(res)). To. List();

IEnumerable<Record> records = Get. Records(); List<Result> results = new List<Result>(); int num. Workers = Environment. Processor. Count; int active. Workers = num. Workers; IEnumerator<Record> enumerator = records. Get. Enumerator(); Optimal concurrency? Manual. Reset. Event completion. Event = new Manual. Reset. Event(false); Must manage event lifetime for(int i = 0; i < num. Workers; i++) { Thread. Pool. Queue. User. Work. Item(delegate { // launch P workers Scalability bottleneck while(true) { Record current. Record; lock (enumerator) { // grab a record under the lock before processing it if (!enumerator. Move. Next()) break; current. Record = enumerator. Current; } Result res = Process. Record(current. Record); // non-trivial computation per record if (Filter(res)) // check if result matches our criteria { lock(results) results. Add(res); // if so add it into results list under a lock } } if (Interlocked. Decrement(ref active. Workers) == 0) completion. Event. Set(); } completion. Event. Wait. One(); // wait until all workers are done What if this throws? Scalability bottleneck Easy to introduce races Caller thread not utilized

SAME CODE PARALLELIZED USING PLINQ // Parallelized LINQ query List<Result> results = records. As. Parallel(). Select(rec => Process. Record(rec)). Where(res => Filter(res)). To. List();

for (int i = 0; i < 100; i++) Do. Work(i); Parallel. For(0, 100, i => Do. Work(i)); foreach (Record rec in src) Do. Work(rec); Parallel. For. Each(src, rec => Do. Work(rec)); { // independent statements A(); B(); C(); Parallel. Invoke( () => A(), () => B(), () => C()); } Record[] results = source. Select(rec => Transform. Func(rec)). To. Array(); Record[] results = source. As. Parallel(). Select(rec => Transform. Func(rec)). To. Array();

Visual Studio IDE . NET Framework 4 Parallel LINQ Parallel Debugger Task Parallel Library Concurrent Collections Task Scheduler Concurrency Visualizer Sync Primitives Thread Pool CLR

PARALLEL LOOPS Parallel. For(0, 100, i => Do. Work(i)); Parallel. For. Each(src, rec => Do. Work(rec)); Thread 1 i=0 i=1 i=C+2 Thread 1 i=2 rec 1 Thread 2 rec P+1 rec 2 P+1 i=C+3 src Thread P i=P*C+1 i=P*C+2 Thread P rec 2 P rec 3 P

// Breaking from a parallel loop Parallel. For(0, 1000, (i, state) => { if (Search. Func(i)) state. Stop(); } // Controlling Degree Of Parallelism Parallel. Options po = new Parallel. Options() { Max. Degree. Of. Parallelism = 4 }; Parallel. For. Each(source, po, element => Process. Record(element) } // Using thread local state Dictionary<Guid, int> counts = Get. Global. Counts(); Parallel. For. Each(enum. Source, () => { //once per worker thread local init return new List<Result>(); }, (record, loop. State, thread. Local) => { // actual loop body var result = Process. Record(record); if (result. Interesting. Flag) thread. Local. Add(result); //cache results }, (thread. Local) => { //end of loop, once per worker combine delegate lock (counts) foreach (Result res in thread. Local) counts[res. GUID] += res. Count; });

PARALLEL LINQ

HOW PLINQ WORKS (from x in src. As. Parallel() where f(x) select y(x)). Sum(); Thread 1 where f(x) src select y(x) Sum() Thread P where f(x) select y(x) Sum()

PARALLEL LINQ (cont’d)

Enqueue Dequeue T 3 T 2 T 1 Global Queue (FIFO) Thread 1 Dispatch Loop Thread 2 Dispatch Loop Thread N Dispatch Loop

Enqueue Thread 1 Dispatch Loop Dequeue Thread 2 Dispatch Loop Thread N Dispatch Loop T 3 T 2 TT 14 Global Queue (FIFO) Dequeue Enqueue T 5 T 6 T 7 T 8 Steal Thread 1 Local Queue (LIFO) Steal Thread 2 Local Queue (LIFO) Steal Thread N Local Queue (LIFO)

Thread-safe, scalable collections Synchronization Partitioning Initialization Cancellation

// Producer consumer pattern Blocking. Collection<string> bc = new Blocking. Collection<string>(); // Start the producer Task t 1 = Task. Factory. Start. New(() => { while(!stream. Reader. End. Of. Stream) bc. Complete. Adding(); }); bc. Add(stream. Reader. Read. Line()); // Start the consumer Task t 2 = Task. Factory. Start. New(() => { try { // Consume from the blocking collection while (true) Console. Write. Line(bc. Take()); } catch (Invalid. Operation. Exception) { // IOE thrown from Take() indicated completed collection Console. Write. Line("That's All!"); } });

http: //www. microsoft. com/visualstudio/en-us/ http: //blogs. msdn. com/b/somasegar/ http: //msdn. com/data http: //blogs. msdn. com/adonet http: //blogs. msdn. com/astoriateam http: //blogs. msdn. com/efdesign

www. microsoft. com/teched www. microsoft. com/learning http: //microsoft. com/technet http: //microsoft. com/msdn

Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31 st http: //northamerica. msteched. com/registration You can also register at the North America 2011 kiosk located at registration Join us in Atlanta next year