Optimal Scheduling of File Transfers with Divisible Sizes

Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths Mugurel Ionut Andreica Polytechnic University of Bucharest Computer Science Department Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

Summary Motivation – Scheduling File Transfers Ø Offline Scheduling Ø l l Ø maximum profit scheduling (files with divisible sizes) – heuristic algorithm minimum cost scheduling – optimal algorithm Online Resource Management l algorithmic framework for the block partitioning method (=data structure) Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 2

Motivation Ø world-wide development and deployment of distributed systems, services and applications Ø competition for bottleneck resources => poor performance Ø inefficient usage of available resources => poor performance Ø scheduling techniques => performance improvements Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 3

Maximum Profit Scheduling Ø n files (divisible sizes) l l Ø szi – size of the ith file sz 1≤sz 2 ≤. . . ≤ szn szi=qi·szi-1, qi≥ 1 is an integer pi=profit of the ith file (transfer request) k paths l l path j – available in the interval [0, Tj] unit speed schedule the requests non-preemptively + at most one request per path per time unit Ø equivalent to multiple knapsack with divisible item sizes Ø Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 4

Multiple Knapsack with Divisible Item Sizes Ø NP-hard l l can be solved in O(n·max{Tj}k) (extension from single knapsack problem) heuristics • Greedy 1 Multiple. Knapsack(n, k) l l l pack the items optimally in the first knapsack remove the q packed items and the knapsack call Greedy 1 Multiple. Knapsack(n-q, k-1) with the remaining items and knapsacks • Greedy 2 l l Ø sort the items according to some criterion (e. g. Profit/size) insert the items into knapsack using the First Fit heuristic new heuristic algorithm – this paper Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 5

New Heuristic Algorithm (1/2) (Multiple Knapsack with Divisible Item Sizes) Ø split the items into groups l l Ø within a group, sort the items in decreasing order of their profits l Ø all the items in the same group (i) have the same size (sgi) sg 1>sg 2>. . . >sg. G (G=total # of groups) pri, 1≥pri, 2≥…≥pri, ni (ni=the # of items in group i) insert the items into knapsacks using the First Fit heuristic l l traverse the items in inreasing order of group number; within a group, in increasing order of item number choice of knapsack for an item (i, j) – not important => because of divisible item sizes, we insert the same set of items Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 6

New Heuristic Algorithm (2/2) (Multiple Knapsack with Divisible Item Sizes) Ø repeatedly improve the initial solution l l Ø replace an item (i, j) in the knapsack with a subset Q of items outside the knapsack, s. t. pr(i, j)<profit(Q) && profit(Q)-pr(i, j)=maximum ignore item (i, j) from now on time complexity l l O(n·S·min{n, S·log(S)}) S=maximum size of an item Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 7

Performance Evaluation (Multiple Knapsack with Divisible Item Sizes) Ø tested algorithms l l Ø new heuristic Greedy 1 Multiple. Knapsack Greedy 2 (with multiple criteria) direct solution (extension from single knapsack) results l l most of the cases – solved optimally by the new heuristic in terms of quality of solution and running time => the new heuristic = clear winner • followed by Greedy 1 Multiple. Knapsack Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 8

Minimum Cost Scheduling (1/3) Ø sequence of n identical files l l Ø to be sent consecutively from the same source to the same destination sending time per file=1 time unit k data transfer providers l provider j • • cost: Cj per time unit lease at most ONE time interval of duration at most Tmax, i which includes [T 1, j, T 2, j] default network link: cost Li for transferring the ith file Ø minimize the total cost Ø Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 9

Minimum Cost Scheduling (2/3) O(n·k) dynamic programming sort the data transfer providers in increasing order of T 2, j: T 2, 1≤T 2, 2≤…≤T 2, n Ø Cmin[i, j]=the minimum total cost for sending the first j files using a subset of the first i providers (in the sorted order) Ø Ø Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 10

Minimum Cost Scheduling (3/3) Ø compute an auxiliary array minpi in O(n) time Ø Ø for j in [T 2, i, T 1, i+Tmax, i]: l Ø Ø Cmin[i, j]=min{Cmin[i-1, j], Cmin[i, j-1]+Lj, (j-T 1, i)·Ci+minpi[j-Tmax, i]} Cmin[k, n] – the answer O(n·k) overall Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 11

Online Resource Management (1/4) Ø scenario l l Ø a resource manager receives resource allocation and reservation requests request=amount of resource (bandwidth) + time constraints (fixed duration, earliest start time, latest finish time) model l l time – divided into equally-sized time slots many requests simultaneously => we need low response times => efficient data structure Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 12

Online Resource Management (2/4) Ø Ø algorithmic framework block partitioning method l l l Ø point and range queries l l Ø array of n cells each cell – value vi divide the cells: n/k blocks of size k update and query functions: O(k+n/k) k=sqrt(n) => O(k+n/k)=O(sqrt(n)) vi=? q. Func(va, va+1, . . . , vb)=? point and range updates l l vi=u. Func(u, vi), a≤i≤ b Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 13

Online Resource Management (3/4) Ø update functions l l l Ø BPpoint. Update BPrange. Update. Points BPrange. Update. Partial. Block BPrange. Update. Full. Block query functions l l l BPpoint. Query BPrange. Query. Points BPrange. Query. Partial. Block BPrange. Query. Full. Block Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 14

Online Resource Management (4/4) Ø solves efficiently many problems l l l Ø range addition update+range min query range set update+range maximum segment query. . . many more typical resource reservation requests l l range addition update (reserve bandwidth for a full time interval) range minimum query (check to see if enough bandwidth is available in every time slot) Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 15

Conclusions Ø offline scheduling problems l maximum profit scheduling • multiple knapsack with divisible item sizes – efficient heuristic l minimum cost scheduling • optimal dynamic programming algorithm Ø online resource management l algorithmic framework (new) for the block partitioning method (well known) Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 16

Thank You ! Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 17