Yardstick A Benchmark for Minecraftlike Services Jerom van

Yardstick A Benchmark for Minecraft-like Services Jerom van der Sar Jesse Donkervliet Alexandru Iosup Contact: opencraft@atlarge-research. com www. atlarge-

Minecraft = Block-based Online Game Blocks © Mojang and Microsoft 2

Minecraft = Most Popular Block-based Online-game 1. 50 million active users 2. Over 100 million sales worldwide a. 53, 000 daily b. 20 since this presentation started 3. Mojang valued at $1. 8 billion 4. Microsoft bought for $2. 5 billion Sources: techspot. com 3

Minecraft = Most Popular Block-based Online Virtual Environment Also: Only game in top-20 by sales for Nintendo Wii/Switch Source: arstechnica. com 4

Minecraft = Uniquely Modifiable Online Game 1. Terrain is completely modifiable a. No baked lighting / light maps b. Unstable 3 D pathfinding c. No static maps ⇒ More network 2. Terrain influences computation a. Redstone ⇒ Computation in MC b. Physics ⇒ Falling blocks, liquid flow, arrows, etc. Sources: curseforge. com, codingame. com, minecraftsix. com 5

Minecraft = Subject to Performance Optimization Undocked Docked (Battery) (A/C) Minecraft on Nintendo Switch: docked (A/C) vs undocked (Battery) Source: techspot. com 6

1. Community is booming a. Independently hosts 50 k+ servers b. 10+ server distributions c. 38 k custom modules 2. Minecraft offered ‘as a Service’ a. Unofficially: 50+ providers b. Officially: Mojang Realms Sources: bstats. org, curseforge. com Minecraft servers: 7

Which Minecraft-like Server Performs ‘Best’? 1. Community is booming Minecraft servers: a. Independently hosts 50 k+ servers b. 10+ server distributions 38 k to custom modules Q: c. How evaluate the performance of Minecraft-like servers? 2. Minecraft offered ‘as a Service’ Unofficially: providers games scale to millions? Q: a. How to make 50+ Minecraft-like b. Officially: Mojang Realms Sources: bstats. org, curseforge. com 8

Opencraft: An Umbrella Project Yardstick A benchmark for Minecraft-like services Meerkat A prototype to explore novel scalability techniques for Minecraft 9

Exploring Minecraft-like Services 10

Intermezzo: A System Model for Minecraft-like Services 11

Intermezzo: Game Loops Spent processing new game state Thread sleep 12

Intermezzo: Game Loops Spent processing new game state Thread sleep Q: How to measure performance? A: Define a metric! 13

Intermezzo: Game Loops 14

Yardstick A Benchmark for Minecraft-like Services 15

Yardstick: Goals 1. Performance evaluation tool for Minecraft-like games 1. Goals: a. Benchmark & compare server performance: Which architecture and/or implementation for Minecraft-like games performs ‘best’? b. Analyze server performance: Identify bottlenecks and performance issues 16

Yardstick: Design Overview 17

Yardstick: Experimental Setup Experiments performed on DAS-5 1. Experimental infrastructure in NL 2. 7 x bare-metal machines with a. Intel E 5 -2630 v 3 CPUs b. 64 GB RAM c. Infiniband networking 3. 1 x Minecraft, 1 x Monitoring, 5 x Yardstick emulator Workload consist of players that alternate between traveling (walking a large distance) and interacting (walking many short distances) 18

Yardstick: Network traffic (out) ≈ 160 Mbps outgoing 19

Yardstick: Network traffic (out) ≈ 160 Mbps outgoing Minecraft-like services use a lot of network traffic 20

Yardstick: Tick Utilization ≈ 100% tick utilization 21

Yardstick: Tick Utilization ≈ 100% tick utilization Glowstone performs the worst 22

Yardstick: Update Frequency ≈ 125 players ⇒ freq. drops 23

Yardstick: Update Frequency ≈ 125 players ⇒ freq. drops Minecraft only scales to hundreds of players 24

Yardstick Results: CPU Utilization Never uses all 16 cores! 25

Yardstick Results: CPU Utilization Q: Why does tick freq. drop when not all cores are used? A: Minecraft-like services are poorly parallelized! Never uses all 16 cores! 26

Wrapping Up 27

Yardstick: Future Work Future work: 1. Realistic player emulation based on real-world message dumps 2. Support for all major server implementations 3. Complex interactions with the virtual environment 28

Take-Home Message 1. 2. 3. 4. Minecraft is a unique and interesting online game Performance of Minecraft-like services is unclear Yardstick = Benchmark targeted at Minecraft-like games Minecraft-like services. . . a. scale to hundreds of players b. are poorly parallelized c. use a lot of network to transfer the terrain 29

Talk To Us at These Places: Yardstick @Large Research ● Delft University of Technology & Vrije Universiteit Amsterdam ● www. atlarge-research. com/opencraft ● opencraft@atlarge-research. com Jerom van der Sar ● J. vander. Sar@atlarge-research. com Jesse Donkervliet ● J. Donkervliet@atlarge-research. com Alexandru Iosup ● A. Iosup@vu. nl (So long, and thanks for all the fish) 30