Graal VM Efficient LanguageLevel Virtualization Eric Sedlar VicePresident

Graal. VM Efficient Language-Level Virtualization Eric Sedlar Vice-President & Technical Director, Oracle Labs Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

We live in a multilingual computing world Long tail of programming languages PROGRAMMING LANGUAGE POPULARITY (TOP 20 LANGUAGES FROM MAY 2017 TIOBE INDEX) • If you run more than a few apps you probably run more than a few languages • We use different languages for different jobs: – R or Python for data science – Ruby or Java. Script for front ends – Java or C/C++ for server back ends Java Others C C++ • Unfortunately, most runtimes can’t run most languages very well – JVM doesn’t run JS, Python or native code well – CLR doesn’t run languages not from MSFT very well Python C# Scratch PL/SQL VB. NET Delphi/Object Pascal MATLAB Go Objective-C Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Java. Script R Swift VB Ruby Perl PHP Assembly language 2

Graal. VM is a meta-runtime for language virtualization Doesn’t just run apps—designed to run things that run apps (i. e. interpreters) • Graal. VM languages are built on the “Truffle” API (in Java) for constructing interpreters – Interpreters define the semantics of the language, and are also used to profile program behavior to guide compilation of frequently-used code (e. g. loops) • Graal. VM includes multiple language interpreters, and learns the type system and language from what they do in order to generate an efficient compiler based on that interpreter • Since the interpreters are all implemented in Java, merging them together into a single engine looks the same to Graal. VM like a single (very complicated) language (with zero language crossing costs) Graal compiler Java. Script Interpreter R Python Ruby Interpreter Truffle API Graal watches Truffle API activity to learn the language Java Runtime Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Compiled code snippet Compiled code snippet 3

Let’s fill in the complete Graal. VM architecture (I) Step one: run each language’s application, libraries & frameworks on the interpreters • Looks easy, right? Graal compiler Java. Script application Java. Script libraries Ruby application Ruby gems & libraries Node. js Rails Java. Script Interpreter … R Python Ruby Interpreter Truffle API Java Runtime Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Compiled code snippet Compiled code snippet 4

But many language runtimes have a lot of native libraries They use native code for better performance in core areas or for systems functionality • If you run native code in Graal. VM, doesn’t that break all of the abstraction you want from a virtual machine? • Is that a secure way to run applications? Native code isn’t in a safe sandbox Graal compiler Java. Script application Java. Script libraries Ruby application Ruby gems & libraries Node. js Rails Java. Script Interpreter … R Python Ruby Interpreter Truffle API Java Runtime Native Libraries & OS ? ? ? Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Compiled code snippet Compiled code snippet 5

We have a solution for that—another Truffle interpreter LLVM (“low level virtual machine”) bitcode interpreter • LLVM supports dozens of (static) languages (C, C++, FORTRAN, COBOL, Rust, Julia, Swift, Objective-C, …) • The LLVM compiler generates “bitcode” (i. e. a. bc file) for all of the languages as an intermediate step • We can write a Truffle interpreter for LLVM bitcode and run in the (bounds-checked) VM sandbox! Graal compiler Java. Script application Java. Script libraries Ruby application Ruby gems & libraries Node. js Rails Java. Script Interpreter … R native libraries Node. js native libs Ruby/Rails native libs LLVM. bc R Python Ruby Interpreter Truffle API Java Runtime Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Compiled code snippet Compiled code snippet 6

This picture of the Graal. VM architecture is simplified The parts look a bit different depending on where Graal. VM is embedded • Notice we aren’t showing how to run Java apps yet? Graal. VM compiler Java. Script application Java. Script libraries Ruby application Ruby gems & libraries Node. js Rails Java. Script Interpreter … R native libraries Node. js native libs Ruby/Rails native libs LLVM. bc R Python Ruby Interpreter Truffle API Java Runtime Threads / Scheduler Filesystem Network GC / heap Compiled code snippet Compiled code snippet Code cache 8 Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Graal. VM embedded in the Java VM The parts look a bit different depending on where Graal. VM is embedded • When Graal. VM is embedded in the Hot. Spot JVM, you can run Java apps directly on the JVM without Truffle • In this case, JVM uses Graal as a Java bytecode compiler via the JVM Compiler Interface (JVMCI, or JEP 243) – Note: Truffle gives an “Abstract Syntax Tree” (language-agnostic representation of work to do) to Graal, not Java bytecodes eval("application/x-ruby", "def add(a, b) a + b; end; "); Polyglot API Ruby application Ruby gems & libraries Node. js Rails Java. Script Interpreter Truffle CI Graal compiler Java. Script application Java. Script libraries … R native libraries Node. js native libs Ruby/Rails native libs LLVM. bc R Ruby Interpreter Truffle API Java application Java. VM Runtime JVMCI Threads / Scheduler Filesystem Network GC / heap Code cache Compiled code snippet Compiled code snippet 9 Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Graal. VM embedded in the Oracle Database Embedding Graal. VM allows substitution of the underlying engine’s service layer • Oracle database takes on many functions of an OS: scheduling, resource management, I/O – Graal. VM allows mixing & matching system components from the underlying runtime eval("application/x-ruby", "def add(a, b) a + b; end; "); Truffle Polyglot API Java. Script application Java. Script libraries Ruby application Ruby gems & libraries Node. js Rails Java. Script Interpreter LLVM. bc R Ruby Interpreter Truffle CI Graal compiler … R native libraries Node. js native libs Ruby/Rails native libs Truffle API Graal. VM Runtime GC / heap Oracle RDBMS Runtime Net Code cache Filesystem Threads / Scheduler Compiled code snippet Compiled code snippet 10 Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Why is Graal. VM Different? • Multilingual • Embeddable • Secure • Efficient Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 11

PROGRAMMING LANGUAGE POPULARITY (TOP 20 LANGUAGES FROM MAY 2017 TIOBE INDEX) Graal. VM Language Status [CATEGORY NAME] Available 2017 Available 2018 Groovy FORTRAN Kotlin Clojure Scala F# Others C Proposed / Prototyping C++ Python The rest of the languages are not really applicable on a server, or are legacy (except Go) C# Scratch PL/SQL VB. NET Delphi/Object Pascal MATLAB Go Objective-C Java. Script R Assembly language Swift Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | VB Ruby Perl PHP 12

Most language-level runtimes are bad at multilingualism What are the issues? • Language-level runtimes generally designed with a one or two languages in mind – The semantics of the original language pollute the VM, so other languages don’t fit well • Foreign languages running on a language-level VM generally have a hard time achieving high degrees (99%+) levels of compatibility • Performance of multilingual runtimes is often poor outside of the base language – JDK Nashorn is 3 x – 5 x slower than Google V 8, Apple Safari • Generally a performance penalty for crossing language barriers (e. g. JNI) • Running multiple separate VMs means two separate heaps to size, to separate engines to tune & manage, and 10 x-1000 x higher overhead to cross language boundaries Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 13

Multilingual Demo • Combining languages in one runtime Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 14

Embeddability • Graal. VM allows any portions of the runtime (memory management, thread scheduling, code cache, etc. ) to be supplied by the embedding engine • We’ll demonstrate this in tomorrow’s session, “Accelerating Your Database Applications Simply with Graal” (using the Oracle RDBMS & My. SQL) Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 15

Graal. VM is a hybrid of static & dynamic runtimes Substrate. VM provides standalone & embeddable infrastructure to run Graal. VM • Application code can be precompiled or dynamically loaded via SVM Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential Oracle Internal/Rstricted/Highly Restricted 16

Flexibility and security with Graal. VM two-layer design Host Language (Java + native) Guest Language (Ruby, Java. Script, Python, R, Java, native) • Ahead-of-time compiled (closed world) • Dynamically compiled (open world) – 100 x faster startup – 2 x lower footprint * – Runtime-profiled against the shape of actual data to maximize peak performance – Add in new code to running system • Still managed (garbage collected, bounds • Flexible, productive (dynamic) checked, & secured against stack overflows) • Memory-boxed & time-boxed so that totally untrusted code can run here • Trusted code should run here • Even native extensions to guest language • Only whitelisted code is guest accessible libraries can be run safely – Reflection is limited to white lists as well – No language-crossing overhead for extensions Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

High-Performance for Each Language Speedup, higher is better 5 4. 1 4. 5 Graal 4 Best Specialized Competition 3 2 1 1. 02 1. 2 0. 850000001 0. 9 0 Java Scala Ruby R Native Java. Script Performance on 64 -bit x 86 on well-known benchmark suites relative to: Hot. Spot/Server, JRuby, GNU R, LLVM AOT compiled, V 8 Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 18

Application Level Performance What about real applications, not just benchmarks • According to Twitter, Graal runs Twitter 25% faster than Java 8 • Web. Logic Server has 10% better throughput running basic web pages Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 19

Multiplicative Value-Add of Graal Ecosystem Your Code Languages Java. Script Ruby R Python C/C++. FORTRAN, … * Graal * Optimizations Tooling Interoperability Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Embeddings Hot. Spot JVM My. SQL Oracle RDBMS Spark nginx …

Graal. VM Community • Currently we use graal-dev@openjdk. java. net for all external discussions – We want to move most traffic off the Open. JDK site other than information related to integration of the Graal compiler with Open. JDK • New mailing list: graalvm-users@oss. oracle. com • Web: otn. oracle. com/oracle-labs/program-languages/overview/index. html • These demos were done with graalvm-0. 28 • Graal projects on github: – github. com/graalvm Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 21

Possible University Collaborations with Graal. VM One thing academia does better than industry is evaluate developer usability • Graal. VM’s Poly. Glot API allows foreign language objects to be used in other languages in a natural way (e. g. field accesses, etc. ) to the extent possible – Are there confusing semantics for multilingual interop? Where? – Let’s build some applications taking advantages of mixing/matching libraries from multiple languages • Truffle makes writing a new language easy & efficient – It also solves the biggest problem with domain-specific languages, which is what happens when you reach the edge of your domain – Interesting DSL explorations for Truffle & Graal. VM? Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 22

23