A High Performance Middleware in Java with a

Outline • Motivations and Overview • Jem 3 D: 3 D electromagnetism • Pro.

Motivations • Offer an integrated solution for – Programming – Deploying – Running applications

Jem 3 D • Solves 3 D Maxwell equations • Relies on finite volume

Jem 3 D (2) Initialization Compute mag. field, and update elec. field Compute elec.

Distribution of the Computation Example on 4 nodes N 1 2 1 1 2

Pro. Active • Java library for parallel, distributed and concurrent programming • Medium grain

Group Communications • Idea / Goals – Manipulate groups as ‘simple’ Java objects –

Deployment Descriptor • Abstract Away from source code: – Machines names – Creation/Connection Protocols

Deployment Descriptor- Examples Virtual. Node: jem 3 d. Node Mapping: jem 3 d. Node

Ibis • Grid programming environment with efficient communications • Java Implementation for portability •

Improved RMI implementation • Source compatible with Sun’s RMI (replace java. rmi. * with

Running Environment • DAS-2 Multi. Cluster – – – 200 Nodes on 5 sites

Comparison with a Fortran Implementation • Not a Java vs Fortran/MPI Comparison! • We

Running Environment • Cluster located in Sophia-Antipolis – 16 nodes of Pentium 3 at

Sequential Version Time Memory Java/Fortran Mesh Java Fortran Time Memory 21 x 21 45

Conclusion • Pro. Active + Ibis: – Integrated Solution – Allows a better speedup

Resources • Jem 3 D: not available yet, but working on it • Pro.

Slides: 29

Download presentation

A High Performance Middleware in Java with a Real Application Fabrice Huet*, Denis Caromel*, Henri Bal+ * Inria-I 3 S-CNRS, Sophia-Antipolis, France + Vrije Universiteit, Amsterdam, Netherlands

Outline • Motivations and Overview • Jem 3 D: 3 D electromagnetism • Pro. Active: High Level Middleware for the Grid • Ibis: High Performance Communications • Java experimentations • Comparison with a Fortran Implementation • Conclusion

Motivations • Offer an integrated solution for – Programming – Deploying – Running applications in an heterogeneous environment • Investigate the current situation of Java programming for high performance

Overview

Jem 3 D • Solves 3 D Maxwell equations • Relies on finite volume approximation method • Works on unstructured tetrahedral discretization of the computation domain • In this paper, we compute the propagation of an eigenmode in a cubic metallic cavity (standard test case) • The complexity of the computation is linked to the number of tetrahedra • We set the number of points to be used for tetrahedra (Mesh size)

Jem 3 D (2) Initialization Compute mag. field, and update elec. field Compute elec. field, and update mag. field Calculation of the discrete electromagnetic energy t < tmax t = tmax Solution Saving Communication intensive

Distribution of the Computation Example on 4 nodes N 1 2 1 1 2 4 Communication Area: 3*N 2 Communication Area: 2*N 2

Pro. Active • Java library for parallel, distributed and concurrent programming • Medium grain entities: active object – Has its own thread of execution – Stores and serves requests – Asynchronous communications with transparent future objects and wait by necessity • Uses RMI or Soap-like protocol • High level features: mobility, security, component, group communication, deployment descriptors…

Group Communications • Idea / Goals – Manipulate groups as ‘simple’ Java objects – Maintain the ‘dot’ notation – Code reusability • Remote method invocation on a group • Multi-unicast approach – Based on the Pro. Active communication mechanism – Replication of n ‘single’ communication – Uses of multiple threads

Deployment Descriptor • Abstract Away from source code: – Machines names – Creation/Connection Protocols – Lookup and Registry Protocols • Interfaced with various protocols and Infrastructures: – Cluster: LSF, PBS, SGE , OAR and PRUN(custom protocols) – Internet and LAN: rsh, rlogin, ssh – Grid: Globus, Web services, gsissh • API to query resources from within the application through symbolic names (Virtual. Node)

Deployment Descriptor- Examples Virtual. Node: jem 3 d. Node Mapping: jem 3 d. Node VM 1, VM 2 localhost Infrastructure: VM 1 Local VM App. host 1 VM 2 jem 3 d. Node VM 2 SSH host 1 then Local VM Virtual. Node: jem 3 d. Node Mapping: jem 3 d. Node VM 1 Infrastructure: VM 1 SSH frontend then pbs. Process PBS 10 nodes Local VM localhost VM VM VM frontend App. jem 3 d. Node

Ibis • Grid programming environment with efficient communications • Java Implementation for portability • Native Implementation for performance – TCP – Myrinet – MPI • Various programming models: Group Method Invocation (GMI), Divide and Conquer (Satin), Remote Method Invocation (RMI) • We focus here on the Java implementation for portability reasons

Improved RMI implementation • Source compatible with Sun’s RMI (replace java. rmi. * with ibis. rmi. *) • Reduced overhead through caching of type information • Overall, reduce by 10% the amount of data sent over the wire • Improved serialization – Generation of serialization code for all classes – Specially generated constructor to create empty objects

Running Environment • DAS-2 Multi. Cluster – – – 200 Nodes on 5 sites ≥ 1 GB of memory We use 72 Nodes on 1 site, other evenly distributed Intra domain nodes linked by 100 Mb/s ethernet Inter domains linked with 1 Gb/s • OS – Red. Hat 7. 2 • Java – Sun’s 1. 5. 0 and IBM’s 1. 4. 1

Jem 3 D with 51 x 51 mesh

Comparison with a Fortran Implementation • Not a Java vs Fortran/MPI Comparison! • We compare: – The same algorithm – Implemented in very different languages (dynamic data structures in Java, static in Fortran) – By different persons – With different goals • We want to know – How much we pay for the Java features – How much we can gain with Ibis

Running Environment • Cluster located in Sophia-Antipolis – 16 nodes of Pentium 3 at 1 Ghz – 512 MB of RAM – Nodes linked with 100 Mb/s Ethernet • OS – Linux 2. 4. 17 – Sun’s JDK 1. 4. 2 • Fortran/MPI – MPICH-1. 2. 5

Sequential Version Time Memory Java/Fortran Mesh Java Fortran Time Memory 21 x 21 45 s 18. 9 s 78 M 59 M 2. 38 1. 32 31 x 31 150 s 65 s 224 M 164 M 2. 30 1. 36 41 x 41 387 s 156 s 483 M 366 M 2. 48 1. 31

Speedup, 51 x 51

Speedup, 81 x 81

Conclusion • Pro. Active + Ibis: – Integrated Solution – Allows a better speedup than plain RMI implementation – Gives to the application a better control on its environment • Still slower (3 x) than a “close” Fortran/MPI implementation but – Easier to compile – Easier to deploy (Jem 3 D ran on 150 nodes) • Evaluate the native solutions – The native ibis version is installed/managed by an administrator – We compile against the Java classes only

Resources • Jem 3 D: not available yet, but working on it • Pro. Active: Available under LGPL – http: //www-sop. inria. fr/oasis/Pro. Active • Ibis: Available under LGPL – http: //www. cs. vu. nl/ibis • Booth #2345