ACI MD GDS Le middleware pour GDS http

ACI MD GDS Le middleware pour GDS http: //graal. ens-lyon. fr/~diet Eddy Caron - ACI GDS – 22/09/03 1

Plan • • Réservation de ressource dans un ASP hiérarchique Déploiement automatique DIET en P 2 P DIET vs Net. Solve Viz. DIET Communications dans DIET Une application pour GDS Eddy Caron - ACI GDS – 22/09/03 2

Context • One long term idea for Grid computing: renting computational power and memory capacity over the Internet – Very high potential • • Need of Problem Solving Environments (PSEs) Applications need more and more memory capacity and computational power Some proprietary libraries or environments need to stay in place Difficulty of installation for some libraries or applications Some confidential data must not circulate over the net Use of computational servers accessible through a simple interface Need of schedulers • Moreover … – Still difficult to use for non-specialists • Almost no transparency • Security and accounting issues usually not addressed – Often application dependent PSEs – Lack of standards • (CORBA, JAVA/JINI, sockets, …) to build the computational servers Eddy Caron - ACI GDS – 22/09/03 3

RPC and Grid-computing Grid. RPC • A simple idea – – RPC programming model for the Grid Use of distributed collections of heterogeneous platforms on the Internet For applications require memory capacity and/or computational power Task parallelism programming model (synchronous/asynchronous) + data parallelism on servers mixed parallelism • Needed functionality – Load balancing • resource discovery • performance evaluation • Scheduling – – Fault tolerance, Data redistribution, Security, Interoperability, … Eddy Caron - ACI GDS – 22/09/03 4

Grid. RPC Client Request S 2 ! B , C (C er sw An A, AGENT(s) ) S 1 Eddy Caron - ACI GDS – 22/09/03 Op(C, A, B) S 2 S 3 S 4 5

Grid. RPC (con’t) • 5 main components: – Client • submits problems to servers • Gives users’ interfaces – Server • solves problems sent by clients • Runs software – Database • contains dynamic and static information about software and hardware resources – Scheduler • chooses an appropriate server depending of – the problem sent – the information contained in the database – Monitor • gets information about the status of the computational resources Eddy Caron - ACI GDS – 22/09/03 6

DIET - Distributed Interactive Engineering Toolbox • Hierarchical architecture for an improved scalability • Distributed information in the tree • Plug-in schedulers MA MA Master Agent MA MA MA Server front end A LA Eddy Caron - ACI GDS – 22/09/03 Direct connection LA LA 7

FAST - Fast Agent’s System Timer • NWS-based (Network Weather Service, UCSB) • Computational performance – Load, memory capacity, and performance of batch queues (dynamic) – Benchmarks and modeling of available libraries (static) • Communication performance – To be able to guess the data redistribution cost between two servers (or between clients and servers) as a function of the network architecture and dynamic information – Bandwidth and latency (hierarchical) C A Eddy Caron - ACI GDS – 22/09/03 B 8

PIF - Propagate Information Feedback • Algorithm from distributed system research – PIF : Propagate Information Feedback • Two steps – First phase: broadcast phase • Broadcast one message through the tree – Second phase: feedback phase • When the leaf has no descendant feedback message is sent to their parent • When the parent receives the feedback messages from all its descendants, it sends a feedback message to its own parent, and so on Eddy Caron - ACI GDS – 22/09/03 9

PIF and DIET - broadcast phase MA A A LA S 1 S 2 S 3 LA S 4 S 5 S 6 LA S 7 S 8 S 9 S 10 S 11 LA S 12 S 13 S 14 S 15 S 16 1. Broadcast the client’s request 2. Sequential FAST interrogation for each LA 3. Resource reservation Eddy Caron - ACI GDS – 22/09/03 10

PIF and DIET - feedback phase MA A A LA S 4 S 1 S 2 S 3 LA S 4 S 5 S 6 S 12 S 7 S 15 LA S 8 S 9 S 10 S 11 LA S 12 S 13 S 14 S 15 S 16 1. chooses the identity of the most (or list of) ``appropriate'' server(s) 2. unused resources are released Eddy Caron - ACI GDS – 22/09/03 11

PIF and DIET - feedback phase MA S 12 S 4 A A LA S 1 S 2 S 3 LA S 4 S 5 S 6 LA S 7 S 8 S 9 S 10 S 11 LA S 12 S 13 S 14 S 15 S 16 1. chooses the identity of the most (or list of) ``appropriate'' server(s) 2. unused resources are released Eddy Caron - ACI GDS – 22/09/03 12

PIF and DIET - feedback phase S 12 MA A A LA S 1 S 2 S 3 LA S 4 S 5 S 6 LA S 7 S 8 S 9 S 10 S 11 LA S 12 S 13 S 14 S 15 S 16 1. chooses the identity of the most (or list of) ``appropriate'' server(s) 2. unused resources are released Eddy Caron - ACI GDS – 22/09/03 13

Server failure and reactivity MA A A S 2 [DEAD LINE 1] LA S 1 S 2 S 3 S 2 LA S 4 S 5 S 6 S 12 S 7 S 15 LA S 8 S 9 S 10 S 11 LA S 12 S 13 S 14 S 15 S 16 • Take into account server failure and increase the DIET reactivity • Time out at the LA level : – Dead Line 1 = ß 1* Call_FAST_time + ß 2 * nb_server Eddy Caron - ACI GDS – 22/09/03 14

Hierarchical fault tolerance MA A S 7 [DEAD LINE 2] LA S 1 S 2 S 3 S 5 S 6 S 7 S 12 S 7 LA S 4 A S 15 LA S 8 S 9 S 10 S 11 LA S 12 S 13 S 14 S 15 S 16 • No answer after dead line 1 – Dead Line 2 = ß 3* level_tree Eddy Caron - ACI GDS – 22/09/03 15

Simulation: Sim. GRID 2 • Real experiments or simulations are often used to test or to compare heuristics. • Designed for distributed heterogeneous platforms • Simulations can enable reproducible scenarios • Simgrid: a distributed application simulator for scheduling algorithm evaluation purposes. – Event-driven simulation – Fixed or According to a trace values to characterize Sim. Grid resources • Processors • Network links • Simgrid 2 : A simulator built using SG. This layer implements realistic simulations based on the foundational SG and is more applicationoriented. – Simulations are built in terms of communicating agents. Eddy Caron - ACI GDS – 22/09/03 16

The DIET Sim. GRID 2 simulator Eddy Caron - ACI GDS – 22/09/03 17

Evaluation of the PIF scheduler Eddy Caron - ACI GDS – 22/09/03 18

Conclusion and future work • Conclusion – Benefit from distributed system research – Fault tolerance into DIET • Server failure • Branch failure – Resource reservation performs a good Qo. S for client requests – DIET Sim. GRID 2 simulator • Can be reused to validate other algorithms • Future work – Implementation of some tools to guarantee the resource reservation – Integrate NWS trace into the simulator – How to fix deadlines on a given heterogeneous platform ? Eddy Caron - ACI GDS – 22/09/03 19

Plan • • Réservation de ressource dans un ASP hiérarchique Déploiement automatique DIET en P 2 P DIET vs Net. Solve Viz. DIET Communications dans DIET Une application pour GDS Eddy Caron - ACI GDS – 22/09/03 20

Automatic deployment • Problem : Take the right number of components (resources) and place them in right way, to increase the overall performance of the platform. • Motivation : “how to deploy DIET on the grid ? ” • Foundation : Idea given in the article “Scheduling strategies for masterslave tasking on heterogeneous processor grids” by C. Banino, O. Beaumont, A. Legrand Y. Robert. Eddy Caron - ACI GDS – 22/09/03 21

Introduction • Solution : – Generate a new structure by arranging the resources according to the graph, which gives best throughput. – For homogeneous platform, the resources should be arranged in a binary tree type structure. – For heterogeneous platform, more resources should be added by checking the bottleneck in the structure. Eddy Caron - ACI GDS – 22/09/03 22

Deployment Ø Architectural model - Eddy Caron - ACI GDS – 22/09/03 23

Deployment § § § wi (Mflp/s): Computing power of node Pi bij : capacite of link (links are symmetric and bidirectional) Sini : size of incomming request from client Souti : size of outgoing request (response) alphaini : fraction of time for the computation of incomming request alphaouti : fraction of time taken for the computaion of outgoing resuest Eddy Caron - ACI GDS – 22/09/03 24

Operations in steady state Pi bij Pj • Calculation of the throughput of a node Eddy Caron - ACI GDS – 22/09/03 25

Calculation of the throughput of a graph Eddy Caron - ACI GDS – 22/09/03 26

Example: Calculation of throughput of graph 20 12 Min(20, 12) 12 Min(12, 7+7) 9 Min(9, 3+4) 7 3 Eddy Caron - ACI GDS – 22/09/03 7 Min(7, 6+5) 4 6 5 27

Homogeneous Structures All nodes have same computing power and bandwidth link Star Graph 2 Depth Star Graph Binary Tree 2 Chain Graph Eddy Caron - ACI GDS – 22/09/03 28

Homogeneous Structures Ø Simulation results (with 8 nodes) - Eddy Caron - ACI GDS – 22/09/03 29

Homogeneous Structures Ø Simulation results (with 32 nodes) - Eddy Caron - ACI GDS – 22/09/03 30

Homogeneous Structures Ø Simulation results (for Binary graph) - Eddy Caron - ACI GDS – 22/09/03 31

Heterogeneous Networks Eddy Caron - ACI GDS – 22/09/03 32

Throughput of network 25 24 R = 2 30 17. 88 19 15 25 25 0. 31 25 1 40 30 29 0. 88 30 0. 052 25 1 200 1 15 Eddy Caron - ACI GDS – 22/09/03 33

Throughput of network by adding LAs 24 R = 2. 2 R = 2. 65 17. 88 19 1. 05 0. 11 15 29 0. 1167 0. 052 0. 88 1 20 11 20 40 200 1 100 11 1 15 Eddy Caron - ACI GDS – 22/09/03 34

Heterogeneous Network Eddy Caron - ACI GDS – 22/09/03 35

Experimental results • 1 client with n requests – no steady state (MA performed good) • n clients with n requests – no steady state (MA performed good) – Pipeline effect – not enough nodes (clients) • Buffered the requests at MA – a new client implementation to make an effect of steady state – MA failed with 960 requests (due to memory problem) Eddy Caron - ACI GDS – 22/09/03 36

Experimental results Eddy Caron - ACI GDS – 22/09/03 37

Conclusion Ø Ø Select best structure Improve throughput of the network Predict the performance of the structure Can find the effects on performance if different changes are done in the structure configuration Ø Bottleneck is not caused at MA Eddy Caron - ACI GDS – 22/09/03 38

Conclusion Ø Homogeneous : § § § Binary tree type structure is best Number of nodes is proportionate to number of servers Star graph type structure, when nodes are less and servers are more than 60 Ø Heterogeneous : § Find the bottleneck § Improve throughput § Modelizing the DIET Eddy Caron - ACI GDS – 22/09/03 39

Future work Ø Calculate throughput of structures with multi-client and multi-master agents. Ø Dynamic updating with the use of package GRAS Ø Timer addition into the tool to get real value for CORBA implementation of DIET Ø Check the LA and Se. D as the cause of bottleneck Ø Combine scheduling and deployment to increase the performance Ø Validation of work by real deployment. Eddy Caron - ACI GDS – 22/09/03 40

Automatic Deployment : first tool Master Agent : . Name = MA 1 <? xml version="1. 0" standalone="yes"? > #!/usr/bin/perl -w . Binary = /home/ckochhof/work/diet/src/dmat_manips/bin/ma 1 use strict; . IP = 193. 253. 175. 223 <launch> use XML: : DOM; . Local agent = LA 1 LA 2 <masteragent name = "MA 1"> my $parser = XML: : DOM: : Parser->new(); Local Agent : <IP>193. 253. 175. 223</IP> my $file = ‘launch. xml'; . Name = LA 1 <binary>/home/ckochhof/work/diet/src/dmat_manips/bin/ma 1</binary> my $doc = $parser->parsefile ($file); . Parent agent= MA 1 <localagent name = "LA 1"> package Master. Agent; . Binary = /home/ckochhof/work/diet/src/dmat_manips/bin/LA 1 <IP>193. 253. 175. 224</IP> sub Master. Agent: : new { . IP = 193. 253. 175. 224 trace. Level = 1 <binary>/home/ckochhof/work/diet/src/dmat_manips/bin/LA 1</binary> my $class = shift; . servers = Se. D 1 Se. D 2 <server name = "Se. D 1"> my $self = {}; $self->{type} = undef; $self->{name}= undef; $self->{IP}= undef; Server : <IP>193. 253. 175. 226</IP> $self->{local. Agents}= {}; $self->{server} = {}; bless $self, $class; $self; agent. Type = DIET_LOCAL_AGENT . Name = Se. D 1 <binary>/home/ckochhof/work/diet/src/dmat_manips/bin/Se. D 1</binary> } . Parent agent = LA 1 <service>all</service> sub Master. Agent: : funtion{ . binary = /home/ckochhof/work/diet/src/dmat_manips/bin/Se. D 1 name = LA 1 </server> my $self = shift; . IP = 193. 253. 175. 226 <server name = "Se. D 2"> my $agt = shift; . services = all <IP>193. 253. 175. 227</IP> $self->{name}= $agt->get. Attribute('name'); ; Parent name = MA 1 Server : <binary>/home/ckochhof/work/diet/src/dmat_manips/bin/Se. D 2</binary> $self->{type} = "Master. Agent"; . Name = Se. D 2 <service>T: Mat. SUM</service> my @child = $agt->get. Child. Nodes; . Parent agent = LA 1 fast. Use = 1 </server> for(my $i=0; $i<$#child; $i++){ . Binary = /home/ckochhof/work/diet/src/dmat_manips/bin/Se. D 2 </localagent> if ($child[$i]->get. Node. Name == 'IP'){ . IP = 193. 253. 175. 227 <localagent name = "LA 2"> $self->{IP} = $child[$i]->get. Data; } . services = T: Mat. SUM ldap. Use = 0 <IP>193. 253. 175. 225</IP> Local Agent : <binary>/home/ckochhof/work/diet/src/dmat_manips/bin/LA 2</binary> if ($child[$i]->get. Node. Name == 'localagent'){ . Name = LA 2 <server name = "Se. D 3"> nws. Use = 1 my $local_name = $child[$i]->get. Data; . Parent agent= MA 1 <IP>193. 253. 175. 228</IP> my $child_pass = $child[$i]; . Binary = /home/ckochhof/work/diet/src/dmat_manips/bin/LA 2 <binary>/home/ckochhof/work/diet/src/dmat_manips/bin/Se. D 3</binary> my $obj_la_ma = new Local. Agent; . IP = 193. 253. 175. 225 <service>T: Mat. SUM</service> $self->{local. Agents}->{$local_name} = $obj_la_ma->funtion($child_pass, $self); } . servers = Se. D 3 </server> Server : </localagent> if ($child[$i]->get. Node. Name == 'server'){ . Name = Se. D 3 </masteragent> my $server_name =$child[$i]->get. Data; . Parent agent = LA 2 </launch> my $child_pass = $child[$i]; . Binary = /home/ckochhof/work/diet/src/dmat_manips/bin/Se. D 3 my $obj_ser_ma = new Server; . IP = 193. 253. 175. 228 $self->{servers}->{server_name}= $obj_ser_ma->funtion($child_pass, $self); } . services = T: Mat. SUM } return $self; } LA 1. cfg To Do • Script to launch the agents. • Simulations to check the parser • Practical implementation Eddy Caron - ACI GDS – 22/09/03 41

Plan • • Réservation de ressource dans un ASP hiérarchique Déploiement automatique DIET en P 2 P DIET vs Net. Solve Viz. DIET Communications dans DIET Une application pour GDS Eddy Caron - ACI GDS – 22/09/03 42

DIET en P 2 P • Existant – Multi-MA disponible avec connection en JXTA – Docs disponibles – Archive disponible : diet-0. 7_beta-dev-jxta. tgz • TODO list MA – Evaluer les performances MA Connexions JXTA MA – Vérifier le respect des coding standard MA MA – Intégration au CVS DIET A – Briser la contrainte 1 composant JXTA pour 1 composant DIET – Algorithmes “intelligents” LA pour le parcours des MA ? LA LA Eddy Caron - ACI GDS – 22/09/03 43

Plan • • Réservation de ressource dans un ASP hiérarchique Déploiement automatique DIET en P 2 P DIET vs Net. Solve Viz. DIET Communications dans DIET Une application pour GDS Eddy Caron - ACI GDS – 22/09/03 44

DIET vs Net. Solve Scripts de déploiement. Utilisation de CVS pour mettre à jour les fichiers de configuration. clients paraski agents + servers sunlabs gateway_vthd clients ls Eddy Caron - ACI GDS – 22/09/03 45

DIET vs Net. Solve Eddy Caron - ACI GDS – 22/09/03 46

DIET vs Net. Solve • TODO List – – – – Tests avec API asynchrone. 'Multithreader' le client. Amélioration des statistiques (indice de dispersion). Amélioration des scripts de déploiements (fichiers de configuration DIET et omni. ORB). Expliquer les résultats de Net. Solve Expliquer le problème des 40 clients de DIET Tests sur les SPARC Tests icluster 2 ? Eddy Caron - ACI GDS – 22/09/03 47

Plan • • Réservation de ressource dans un ASP hiérarchique Déploiement automatique DIET en P 2 P DIET vs Net. Solve Viz. DIET Communications dans DIET Une application pour GDS Eddy Caron - ACI GDS – 22/09/03 48

Viz. DIET • Chaque Log. Manager collecte les infos de son agent et les envoie au Log. Central situé en dehors de la structure DIET. • Viz. Diet : – outil de visualisation en Java – Interraction sur la plate-forme Eddy Caron - ACI GDS – 22/09/03 49

Viz. DIET 1. 0 • Intégration de Log. Service (Log. Manager/Log. Central) dans les agents DIET • Transfert de messages depuis l'agent par l'intermédiaire du Log. Manager – pas de stockage sur disque • Etude viz. Perf vs viz. DIET – Conclusion : viz. Perf trop éloigné de la structure DIET Eddy Caron - ACI GDS – 22/09/03 50

Viz. DIET 2. 0 • Viz. Diet collectera les infos de la structure DIET par Log. Central et les affichera en temps réel. • Viz. Diet doit pouvoir aussi agir sur la structure en générant des scripts XML (flèche rouge) ou en modifiant des infos XML existantes (flèche bleue) Eddy Caron - ACI GDS – 22/09/03 51

Screenshot Eddy Caron - ACI GDS – 22/09/03 52

Plan • • Réservation de ressource dans un ASP hiérarchique Déploiement automatique DIET en P 2 P DIET vs Net. Solve Viz. DIET Communications dans DIET Une application pour GDS Eddy Caron - ACI GDS – 22/09/03 53

Du neuf dans les communications DIET • Asynchrone : – Finaliser la compatibilité Grid. RPC (erreur, handle, …) – Bug sur le diet_wait_and ? – Fuites mémoires ? (test d’échelle) • Padico. TM – – Compilation/Test Intégration dans DIET en cours Limitation des plate-formes utilisables Bug sur le dechargement des modules (lié à la fonction ‘dlopen’ selon la ‘libc’ utilisée) Eddy Caron - ACI GDS – 22/09/03 54

Plan • • Réservation de ressource dans un ASP hiérarchique Déploiement automatique DIET en P 2 P DIET vs Net. Solve Viz. DIET Communications dans DIET Une application pour GDS Eddy Caron - ACI GDS – 22/09/03 55

Une application pour GDS ? • Gri. PPS : Grid Protein Pattern Scanning – Application de bio-informatique – Pattern scanning • Caractéristiques – Tâches répétitives, courtes mais très nombreuses – Entrées/sorties • fichiers « texte » • De quelques Mos à plusieurs Gos – Demande d’un temps de réponse faible Eddy Caron - ACI GDS – 22/09/03 56