Performance Evaluation of JXTA Communication Layers Mathieu Jan

  • Slides: 48
Download presentation
Performance Evaluation of JXTA-* Communication Layers Mathieu Jan PARIS Research Group Paris, November 2004

Performance Evaluation of JXTA-* Communication Layers Mathieu Jan PARIS Research Group Paris, November 2004

Outline n Overview of JXTA communications layers n Experimental setup and benchmarks used n

Outline n Overview of JXTA communications layers n Experimental setup and benchmarks used n Evaluation of JXTA-J 2 SE n n n Fast-Ethernet Myrinet Evaluation of JXTA-C n n Fast-Ethernet Myrinet 2

Motivations: performance of JXTA? n Papers from Emir Halepovic n n n Pipe service

Motivations: performance of JXTA? n Papers from Emir Halepovic n n n Pipe service only No in-depth explanations about the results No precise details about the experimental setup Mainly JXTA 1. x Complex topologies (relays, etc) n n Not clear what is benchmarked Results from the P 3 project n =~ Xtrem. Web over JXTA 3

4

4

5

5

Project Jux. Test n A set of benchmarks for JXTA-J 2 SE 2. x

Project Jux. Test n A set of benchmarks for JXTA-J 2 SE 2. x n n n Co-written with David A. Noblet n n Focuses of JXTA-J 2 SE Communications layers Uses JXTA Distributed Framework project (JDF) Undergraduate at the University of New Hampshire Source code soon available! (22/11) n n http: //jdf. jxta. org/ Results already on-line 6

JXTA-J 2 SE communications layers JXTA Socket Pipe service TCP, HTTP, etc Endpoint service

JXTA-J 2 SE communications layers JXTA Socket Pipe service TCP, HTTP, etc Endpoint service 7

JXTA-J 2 SE communication layers n JXTA Socket n n Pipe service n n

JXTA-J 2 SE communication layers n JXTA Socket n n Pipe service n n n Java Socket interface Data-stream interface Reliability Pipe ID Unreliable Endpoint service n n Peer ID Unreliable 8

Experimental setup n Nodes n n Networks n n Sun 1. 4. 2_01 -b

Experimental setup n Nodes n n Networks n n Sun 1. 4. 2_01 -b 06 (-server -Xms 256 M -Xmx 256 M) IBM 1. 4. 1 (-Xms 256 M -Xmx 256 M) JXTA-J 2 SE peers via JDF n n Fast-Ethernet (MTU of 1500) Myrinet (GM 2. 0. 11, MTU of 9000) JVMs n n 2. 4 GHz Intel P 4, 1 GB of RAM, linux 2. 4. 22 1 rdv peer and 2 edge peers Benchmarks used n RTT and protocol efficiency 9

Performance Evaluation of JXTA-J 2 SE on Fast-Ethernet

Performance Evaluation of JXTA-J 2 SE on Fast-Ethernet

Raw bandwidth results of JXTA Socket Java socket = 11. 22 MB/s JXTA 2.

Raw bandwidth results of JXTA Socket Java socket = 11. 22 MB/s JXTA 2. 3 = 9. 72 MB/s JXTA 2. 2. 1 = 9. 48 MB/s 11

Raw latency results of JXTA Socket Java socket = < 0. 1 ms JXTA

Raw latency results of JXTA Socket Java socket = < 0. 1 ms JXTA 2. 3 =~ 3 ms JXTA 2. 2. 1 =~ 4 ms 12

Protocol efficiency of JXTA Socket n 4 message elements in a JXTA message n

Protocol efficiency of JXTA Socket n 4 message elements in a JXTA message n ACK_NUMBER n n Endpoint. Router. Msg (XML doc) n n n TCP address Endpoint. Destination. Address n n Used by the endpoint router service 565 bytes! Not even needed! Endpoint. Source. Address n n Message payload + reliability TCP address + Endpoint. Service (Net group) + Endpoint Router 1 byte = 909 bytes sent on the wires! 13

Tunning the output buffer size of JXTA Socket output buffer size = size of

Tunning the output buffer size of JXTA Socket output buffer size = size of a pipe message Java socket = 11. 22 MB/s JXTA 2. 2. 1 (512 KB) = 11. 12 MB/s 14

Raw bandwidth results of unicast pipe Java socket = 11. 22 MB/s JXTA 2.

Raw bandwidth results of unicast pipe Java socket = 11. 22 MB/s JXTA 2. 3 = 9. 59 MB/s JXTA 2. 2. 1 = 10. 74 MB/s 15

Raw latency result of unicast pipe Java socket = < 0. 1 ms JXTA

Raw latency result of unicast pipe Java socket = < 0. 1 ms JXTA 2. 3 =~ 2 ms JXTA 2. 2. 1 =~ 35 ms 16

Protocol efficiency of unicast pipe n 4 message elements in a JXTA message n

Protocol efficiency of unicast pipe n 4 message elements in a JXTA message n n n PAYLOAD Endpoint. Router. Msg (XML doc) Endpoint. Source. Address Endpoint. Destination. Address 1 byte = 877 bytes sent on the wires! 17

JXTA Socket vs unicast pipe (bandwidth) JXTA Socket = 9. 72 MB/s JXTA unicast

JXTA Socket vs unicast pipe (bandwidth) JXTA Socket = 9. 72 MB/s JXTA unicast pipe = 9. 59 MB/s 18

JXTA Socket vs unicast pipe (latency) JXTA Socket =~ 3 ms JXTA unicast pipe

JXTA Socket vs unicast pipe (latency) JXTA Socket =~ 3 ms JXTA unicast pipe =~ 2 ms 19

Removing limit on message size (unicast pipe) Java socket = 11. 22 MB/s JXTA

Removing limit on message size (unicast pipe) Java socket = 11. 22 MB/s JXTA 2. 2. 1 = 11. 14 MB/s 20

IBM JVM vs Sun JVM on unicast pipe JXTA 2. 3 IBM =~ 1.

IBM JVM vs Sun JVM on unicast pipe JXTA 2. 3 IBM =~ 1. 2 ms JXTA 2. 3 SUN =~ 1. 8 ms 21

Raw bandwidth results of endpoint service Java socket = 11. 22 MB/s JXTA 2.

Raw bandwidth results of endpoint service Java socket = 11. 22 MB/s JXTA 2. 3 = 10. 47 MB/s JXTA 2. 2. 1 = 11. 01 MB/s 22

Raw latency results of endpoint service Java socket = < 100 us JXTA 2.

Raw latency results of endpoint service Java socket = < 100 us JXTA 2. 3 IBM =~ 480 us JXTA 2. 3 SUN =~ 730 us Issue 1228: 2 TCP messages for 1 JXTA message! 23

Protocol efficiency of endpoint service n 3 message elements in a JXTA message n

Protocol efficiency of endpoint service n 3 message elements in a JXTA message n n 1 byte = only 300 bytes sent on the wires! n n PAYLOAD Endpoint. Source. Address Endpoint. Destination. Address If we can say only … No Endpoint. Router. Msg message element n n Removed 565 bytes! Removed XML parsing 24

Cost of each layer (bandwidth) 25

Cost of each layer (bandwidth) 25

Cost of each layer (latency) 26

Cost of each layer (latency) 26

Performance Evaluation of JXTA-J 2 SE on Myrinet Or how to desperately reach the

Performance Evaluation of JXTA-J 2 SE on Myrinet Or how to desperately reach the Gb/s!

Experimental setup n « Ethernet emulation » feature of GM n n Same benchmarks

Experimental setup n « Ethernet emulation » feature of GM n n Same benchmarks n n No OS-bypass mode : -( A different IP address when configuring JXTA Focus on bandwidth n Removed limit on message size 28

JXTA Socket bandwidth results Results are not very good! Java socket = 159. 20

JXTA Socket bandwidth results Results are not very good! Java socket = 159. 20 MB/s JXTA 2. 3 = 14. 70 MB/s JXTA 2. 2. 1 = 12. 10 MB/s 29

Tuning the output buffer size of JXTA Socket Above 80 MB/s 30

Tuning the output buffer size of JXTA Socket Above 80 MB/s 30

Unicast pipe bandwidth results Above 1 Gb/s! : -) Java socket = 159. 20

Unicast pipe bandwidth results Above 1 Gb/s! : -) Java socket = 159. 20 MB/s JXTA 2. 3 = 106. 54 MB/s JXTA 2. 2. 1 = 136. 78 MB/s 31

Endpoint service bandwidth results Java socket = 159. 20 MB/s JXTA 2. 3 =

Endpoint service bandwidth results Java socket = 159. 20 MB/s JXTA 2. 3 = 112. 32 MB/s JXTA 2. 2. 1 = 144. 97 MB/s 32

Performance Evaluation of JXTA-C on Fast-Ethernet

Performance Evaluation of JXTA-C on Fast-Ethernet

Short overview of JXTA-C n Needs Apache Portable Runtime (APR) n n Newly added

Short overview of JXTA-C n Needs Apache Portable Runtime (APR) n n Newly added support for TCP communications n n Only pipe service and endpoint service Several bug fixes n n Need to use a J 2 SE rdv peer No JXTA Socket layer n n Peer. Com, Seoul (South Korea) Does not support rdv peers n n Threads, Network, etc Mostly Endpoint. Router. Msg TCP_NODELAY on n Several TCP messages 34

Raw bandwidth results of unicast pipe JXTA-J 2 SE 2. 3 = 11. 1

Raw bandwidth results of unicast pipe JXTA-J 2 SE 2. 3 = 11. 1 MB/s JXTA-J 2 SE 2. 2. 1 = 11. 1 MB/s JXTA-C = 11. 1 MB/s Similar results compared to JXTA-J 2 SE 35

Raw latency results of unicast pipe JXTA-J 2 SE 2. 3 =~ 1. 8

Raw latency results of unicast pipe JXTA-J 2 SE 2. 3 =~ 1. 8 ms JXTA-J 2 SE 2. 2. 1 =~ 35 ms JXTA-C =~ 2 ms 36

Protocol efficiency of unicast pipe n 4 message elements in a JXTA message n

Protocol efficiency of unicast pipe n 4 message elements in a JXTA message n n n PAYLOAD Endpoint. Router. Msg (XML doc) Endpoint. Source. Address Endpoint. Destination. Address 1 byte = 834 bytes sent on the wires! n n JXTA-J 2 SE = 877 bytes + Gateway. Forward tag in Endpoint. Router n n Waiting for Bernard Traversat answear - charset tag for each message element 37

Raw bandwidth results of endpoint service JXTA-J 2 SE 2. 3 = 11. 15

Raw bandwidth results of endpoint service JXTA-J 2 SE 2. 3 = 11. 15 MB/s JXTA-J 2 SE 2. 2. 1 = 11. 19 MB/s JXTA-C = 11. 16 MB/s 38

Raw latency results of endpoint service JXTA-J 2 SE 2. 3 =~ 0. 8

Raw latency results of endpoint service JXTA-J 2 SE 2. 3 =~ 0. 8 ms JXTA-J 2 SE 2. 2. 1 =~ 1 ms JXTA-C =~ 0. 8 ms 39

Protocol efficiency of endpoint service n 3 message elements in a JXTA message n

Protocol efficiency of endpoint service n 3 message elements in a JXTA message n n 1 byte = only 239 bytes sent on the wires! n n n PAYLOAD Endpoint. Source. Address Endpoint. Destination. Address If we can say only … JXTA-J 2 SE = 300 bytes No Endpoint. Router. Msg n n Removing 595 bytes! Removed XML parsing 40

Unicast pipe vs endpoint service 41

Unicast pipe vs endpoint service 41

Performance Evaluation of JXTA-C on Myrinet Still using « Ethernet emulation » feature of

Performance Evaluation of JXTA-C on Myrinet Still using « Ethernet emulation » feature of GM

Unicast pipe bandwidth results JXTA-J 2 SE 2. 3 = 106 MB/s JXTA-J 2

Unicast pipe bandwidth results JXTA-J 2 SE 2. 3 = 106 MB/s JXTA-J 2 SE 2. 2. 1 = 136 MB/s JXTA-C = 107 MB/s 43

Endpoint service bandwidth results JXTA-J 2 SE 2. 3 = 112. 32 MB/s JXTA-J

Endpoint service bandwidth results JXTA-J 2 SE 2. 3 = 112. 32 MB/s JXTA-J 2 SE 2. 2. 1 = 144. 97 MB/s JXTA-C = 109 MB/s 44

JXTA-C on top of Padico. TM : -) n JXTA-C is working on top

JXTA-C on top of Padico. TM : -) n JXTA-C is working on top of Padico. TM! (17/11) n « sed » modifications in apr n n n JXTA-C configured to use GDBM instead of SDBM Goal: stop using « ethernet emulation » of GM … n n Using marcel threads instead of pthread But JXTA-C is not yet zero-copy : -( Performance evaluation in progress … 45

Conclusion n Performance evaluation of the communication layers of JXTA-* (J 2 SE and

Conclusion n Performance evaluation of the communication layers of JXTA-* (J 2 SE and C) n n n Much better results compared to other related work and explained results n n Fast-Ethernet Myrinet Known costs of JXTA-J 2 SE nearly saturate a Fast-Ethernet link JXTA-J 2 SE on Myrinet bandwidth greater than 1 Gb/s JXTA-J 2 SE > JXTA-C on Myrinet n JXTA-C is not zero-copy 46

Ongoing work n n Performance evaluation of JXTA-C on top of Padico. TM Jux.

Ongoing work n n Performance evaluation of JXTA-C on top of Padico. TM Jux. Mem client in C for DIET n n Better knowledge of JXTA-C tricks ; -) Write papers n GP 2 PC (1/12) on JXTA raw performance n n Fast-ethernet ? on JXTA tuned performance for grid computing n Myrinet 47

Future work n Benchmark secure and propagate pipes n Update Jux. Test results n

Future work n Benchmark secure and propagate pipes n Update Jux. Test results n JXTA 2. 3. 1 n Unidirectional throughput test n Plug-in for Ethereal n An easier analysis of JXTA protocols 48