Maximize Your NET Application Performance Task Parallel Library

Maximize Your. NET Application Performance Task Parallel Library - TPL 25 th September 2014 by Romulus Susanu

Topics • Parallel Processing and Concurrency • Synchronous vs Asynchronous • Asynchronous Programming Patterns in. NET • Task Parallel Library - TPL § Data Parallelism § Task Parallelism

Parallel Processing and Concurrency • keep your application responsive • maximize the performance of your code

Parallel Processing vs Concurrency Parallelism • refers to techniques to make programs faster by performing several computation in parallel. • requires hardware with multiple CPU Key problem • reduce data dependencies

Parallel Processing vs Concurrency • refers to techniques that make program more usable • can be implemented and is used a lot on single CPU • multi-tasking operating system is synonym for supporting concurrency

Parallel Processing vs Concurrency

Synchronous vs Asynchronous • in. NET it's directly related with threads • in a threaded system the decision to suspend one thread and execute another is largely outside of the programmer’s control • threaded model execution are handled by the OS

Synchronous vs Asynchronous Synchronous • wait for it to finish before moving on to another task Asynchronous • move on to another task before it finished • executing a process/task on another thread Why asynchronous programming is more performing?

Synchronous vs Asynchronous Why asynchronous programming is more performing? Blocking in a synchronous program Why would a task be blocked? The asynchronous model

Synchronous vs Asynchronous Why would a task be blocked? • waiting to perform I/O • to transfer data to or from an external device. An asynchronous program is often called a non-blocking program!

Synchronous vs Asynchronous When the asynchronous model performs best? • there a large number of tasks so there is likely always at least one task that can make progress. • the tasks perform lots of I/O, causing a synchronous program to waste lots of time blocking when other tasks could be running. • a network server implementation is a prime candidate for the asynchronous model Every time the OS transfers control over from one thread to another it has to save all the relevant registers, memory map, stack pointers, FPU context etc. so that the other thread can resume execution where it left off.

Asynchronous Programming Patterns in. NET • Asynchronous Programming Model (APM) (also called IAsync. Result Pattern) • Event-based Asynchronous Pattern (EAP) • Task-based Asynchronous Pattern (TAP)

Asynchronous Programming Patterns in. NET Asynchronous Programming Model (APM) • asynchronous operations require Begin and End methods • no longer recommended for new development

Asynchronous Programming Patterns in. NET Asynchronous Programming Model (APM) Code sample: • consider a Read method that reads a specified amount of data into a provided buffer starting at a specified offset public class My. Class { public IAsync. Result Begin. Read(byte [] buffer, int offset, int count, Async. Callback callback, object state); public int End. Read(IAsync. Result async. Result); }

Asynchronous Programming Patterns in. NET Event-based Asynchronous Pattern (EAP) • requires a method that has the Async suffix and one or more events • was introduced in the. NET Framework 2. 0 • no longer recommended for new development

Asynchronous Programming Patterns in. NET Event-based Asynchronous Pattern (EAP) Code sample: public class Async. Example { // Synchronous methods. public int Method 1(string param); // Asynchronous methods. public void Method 1 Async(string param); public event Method 1 Completed. Event. Handler Method 1 Completed; public void Cancel. Async(); public bool Is. Busy { get; } } Background. Worker components represent more complex implementations of (Run. Worker. Async(), Cancel. Async, Progress. Changed, Run. Worker. Completed etc. ) EAP

Asynchronous Programming Patterns in. NET Task-based Asynchronous Pattern (TAP) • uses a single method to represent the initiation and completion of an asynchronous operation • was introduced in the. NET Framework 4. 0 • recommended approach to asynchronous programming

Asynchronous Programming Patterns in. NET Task-based Asynchronous Pattern (TAP) Code sample: public class My. Class { public Task<int> Read. Async(byte [] buffer, int offset, int count); } System. Threading. Tasks Namespace • provides types that simplify the work of writing concurrent and asynchronous code

Task Parallel Library - TPL Data Parallelism Task Parallelism

Task Parallel Library - TPL Data Parallelism • refers to scenarios in which the same operation is performed concurrently • the source collection is partitioned so that multiple threads can operate on different segments concurrently • data parallelism through the System. Threading. Tasks. Parallel class • Parallel class provides method-based parallel implementations of for and foreach loops

Task Parallel Library - TPL Data Parallelism The Sum. Root. N method returns the sum of the nth root of all integers from one to 10 million, where n is a parameter Sequential loop for(int i=2; i<20; i++) { var result=Sum. Root. N(i); Console. Write. Line("root{0}: {1}", i, result); } public static double Sum. Root. N(int root) { double result=0; for(int i=1; i<10000000; i++) { result +=Math. Exp(Math. Log(i)/root); } Return result; }

Task Parallel Library - TPL Data Parallelism // For loops // For. Each loops var roots = new List<int> { 2, 3, 4, …, 20}; // Method signature: Parallel. For(int from. Inclusive, int // Method signature: Parallel. For. Each(IEnumerable<TSource> to. Exclusive, Action<int> body) source, Action<TSource> body) Parallel. For(2, 20, (i)=> { var result=Sum. Root. N(i); Console. Write. Line("root{0}: {1}", i, result); }); Parallel. For. Each(roots, (i)=> { var result = Sum. Root. N(roots[i]); Console. Write. Line("root{0}: {1}", i, result); });

Task Parallel Library - TPL Task Parallelism • a task represents an asynchronous operation • in some ways it resembles the creation of a new thread or Thread. Pool work item // Define and run the task. Task task. A = Task. Run( () => Console. Write. Line("Hello from task. A. "));

Task Parallel Library - TPL Task Parallelism Tasks provide two primary benefits: • more efficient and more scalable use of system resources Ø lock-free Ø Hill algorithm - Concurrent. Queue<T> Climbing algorithm Ø Work-Stealing • algorithm more programmatic control than is possible with a thread or Thread. Pool work item

Task Parallel Library - TPL Hill Climbing algorithm • to improve the utilization of cores when threads are blocked by I/O or other wait conditions that stall the processor • the. NET thread pool has an opportunity to inject threads every time a work item completes or at 500 millisecond intervals, whichever is shorter • if adding threads seems to be helping throughput, the thread pool adds more; otherwise, it reduces the number of worker threads.

Task Parallel Library - TPL Work-stealing algorithms Thread. Pool Global Queue vs. Local Queues Extra efficiencies: • improved cache locality • minimized contention

Task Parallel Library - TPL Work-stealing algorithms What happens when a thread's local work queue is empty and the global queue is also empty? Benefits: • Load-balancing

Task Parallel Library - TPL - More programmatic control Creating and Running Tasks Implicitly • just pass in an Action delegate for each item of work Parallel. Invoke(() => Do. Some. Work(), () => Do. Some. Other. Work());

Task Parallel Library – TPL - More programmatic control Creating and Running Tasks Explicitly • a task that does not return a value is represented by the System. Threading. Tasks. Task class • the Task. Wait method to ensure that the task completes execution before the console mode application ends Thread. Current. Thread. Name = "Main"; // Create a task and supply a user delegate by using a lambda expression. Task task. A = new Task( () => Console. Write. Line("Hello from task. A. ")); // Start the task. A. Start(); // Output a message from the calling thread. Console. Write. Line("Hello from thread '{0}'. ", Thread. Current. Thread. Name); task. A. Wait(); // The example displays the following output: // Hello from thread 'Main'. // Hello from task. A.

Task Parallel Library – TPL - More programmatic control Creating and Running Tasks Explicitly Task. Run method • the Run methods use the default task scheduler • The Run methods are the preferred way to create and start tasks when more control over the creation and scheduling of the task is not needed Thread. Current. Thread. Name = "Main"; // Define and run the task. Task task. A = Task. Run( () => Console. Write. Line("Hello from task. A. ")); // Output a message from the calling thread. Console. Write. Line("Hello from thread '{0}'. ", Thread. Current. Thread. Name); task. A. Wait(); // The example displays the following output: // Hello from thread 'Main'. // Hello from task. A.

Task Parallel Library – TPL - More programmatic control Creating and Running Tasks Explicitly Task. Factory. Start. New method Use this method when: • creation and scheduling do not have to be separated • require additional task creation options • use of a specific scheduler Thread. Current. Thread. Name = "Main"; // Better: Create and start the task in one operation. Task task. A = Task. Factory. Start. New(() => Console. Write. Line("Hello from task. A. ")); // Output a message from the calling thread. Console. Write. Line("Hello from thread '{0}'. ", Thread. Current. Thread. Name); task. A. Wait(); // The example displays the following output: // Hello from thread 'Main'. // Hello from task. A.

Task Parallel Library – TPL - More programmatic control Creating and Running Tasks Explicitly System. Threading. Tasks. Task<TResult> class Task<Double>[] task. Array = { Task<Double>. Factory. Start. New(() => Do. Computation(1. 0)), Task<Double>. Factory. Start. New(() => Do. Computation(1000. 0)) }; var results = new Double[task. Array. Length]; Double sum = 0; for (int i = 0; i < task. Array. Length; i++) { results[i] = task. Array[i]. Result; Console. Write("{0: N 1} {1}", results[i], i == task. Array. Length - 1 ? "= " : "+ "); sum += results[i]; } Console. Write. Line("{0: N 1}", sum); } // The example displays the following output: // 606. 0 + 10, 605. 0 + 100, 495. 0 = 111, 706. 0 private static Double Do. Computation(Double start) { Double sum = 0; for (var value = start; value <= start + 10; value +=. 1) sum += value; return sum; }

Task Parallel Library – TPL - More programmatic control Continuation Tasks • in asynchronous programming, it is very common for one asynchronous operation, on completion, to invoke a second operation and pass data to it // The antecedent task. Can also be created with Task. Factory. Start. New. Task<Day. Of. Week> task. A = new Task<Day. Of. Week>(() => Date. Time. Today. Day. Of. Week); // The continuation. Its delegate takes the antecedent task // as an argument and can return a different type. Task<string> continuation = task. A. Continue. With( (antecedent) => { return String. Format("Today is {0}. ", antecedent. Result); }); // Start the antecedent. task. A. Start(); // Use the contuation's result. Console. Write. Line(continuation. Result);

Task Parallel Library – TPL - More programmatic control Task Creation Options var task 3 = new Task(() => My. Long. Running. Method(), Task. Creation. Options. Long. Running); task 3. Start(); Task. Creation. Options parameter value Description None The default when no option is specified. The scheduler uses its default heuristics to schedule the task. Prefer. Fairness Specifies that the task should be scheduled so that tasks created sooner will be more likely to be executed sooner, and tasks created later will be more likely to execute later. Long. Running Specifies that the task represents a long-running operation. Attached. To. Parent Specifies that a task should be created as an attached child of the current task, if one exists. For more information, see Attached and Detached Child Tasks. Deny. Child. Attach Specifies that if an inner task specifies the Attached. To. Parent option, that task will not become an attached child task. Hide. Scheduler Specifies that the task scheduler for tasks created by calling methods like Task. Factory. Start. New or Task<TResult>. Continue. With from within a particular task is the default scheduler instead of the scheduler on which this task is running.

Task Parallel Library – TPL - More programmatic control Continuation Tasks • in asynchronous programming, it is very common for one asynchronous operation, on completion, to invoke a second operation and pass data to it // The antecedent task. Can also be created with Task. Factory. Start. New. Task<Day. Of. Week> task. A = new Task<Day. Of. Week>(() => Date. Time. Today. Day. Of. Week); // The continuation. Its delegate takes the antecedent task // as an argument and can return a different type. Task<string> continuation = task. A. Continue. With( (antecedent) => { return String. Format("Today is {0}. ", antecedent. Result); }); // Start the antecedent. task. A. Start(); // Use the contuation's result. Console. Write. Line(continuation. Result);

Task Parallel Library – TPL - More programmatic control Task Schedulers • makes sure that the work of a task is eventually executed • the default task scheduler is based on the. NET Framework 4 Thread. Pool Custom Task Schedulers Parallel. Extensions. Extras package • Limited. Concurrency. Level. Task. Scheduler - limits the number of threads used by the application • Ordered. Task. Scheduler - guarantees that tasks are processed in the order that they were scheduled. • Thread. Per. Task. Scheduler - dedicates a thread to each individual task

Task Parallel Library – TPL - More programmatic control Real Case foreach (Survey. Detail survey. Detail in survey. Detail. List) { so. Far++; Survey. Processing. Populate. UPSInformation(survey. Detail); } webconfig private static readonly Limited. Concurrency. Level. Task. Scheduler _lcts = new Limited. Concurrency. Level. Task. Scheduler(50); … IList<Task> tasks = new List<Task>(); var task. Factory = new Task. Factory(_lcts); foreach (Survey. Detail survey. Detail in survey. Detail. List) { var verify. By. Ups. Task = task. Factory. Start. New( () => Populate. UPSInformation(survey. Detail)); } <connection. Management> <add address="*" maxconnection="48" /> </connection. Management>

Thank you!