Network Serialization and Routing in World of Warcraft





























































- Slides: 61

Network Serialization and Routing in World of Warcraft Joe Rumsey [email protected] com Twitter: @joerumz

What is JAM? J A M oe’s utomated essages

The Problem Game servers need to communicate with each other

Manual serialization is error-prone void Serialize(stream &msg) { vector<int> values; //. . . Fill in some values. . . msg << values. size(); for(int i = values. size(); --i; ) { msg << values[i]; } } void Deserialize(stream &msg) { vector<int> values; int size; msg >> size; values. resize(size); for(int i = size; i--; ) { msg >> values[i]; } }

Manual serialization doesn’t scale World Of Checkers --server boundary--

Goals • DRY - Don’t Repeat Yourself • Eliminate boilerplate to reduce bugs • No more hand-coded serialize/deserialize • Spend more time on the game, not the protocol • Build a helpful robot that writes our code for us

Goal: Human readable code struct Checker. Captured { Checker. ID id; Checker. ID captured. By; u 8 jump. Type; }; void Capture(Checker. ID id, Checker. ID by, JUMP_TYPE jump. Type) { Checker. Captured msg; msg. id = id; msg. captured. By = by; msg. jump. Type = jump. Type; Send(&msg); }

Implementation Details

Development Cycle • Describe the protocol • Generate serialization and dispatch • Send messages • Receive messages • Configure routing info

1 -to-1 mapping of. jam messages to C++ classes // From Checkers. jam message Checker. Capture. Credit { Checker. ID captured. Checker. ID; Checker. ID captured. By; u 8 jump. Type; }; // 100% Generated code in Jam. Checkers. cpp class Checker. Capture. Credit : public Jam. Message { public: // Message decoders BOOL Get(Binary. Decoder &decoder); BOOL Get(JSONDecoder &decoder); // Message encoders BOOL Put(Binary. Encoder &encoder) const; BOOL Put(JSONEncoder &encoder) const; /**** DATA START ****/ Checker. ID captured. Checker. ID; Checker. ID captured. By; u 8 jump. Type; /**** DATA STOP ****/ // Lots more stuff. . . };

Development Cycle • Describe the protocol • Generate serialization and dispatch • Send messages • Receive messages • Configure routing info

Auto-generated serialization code //NOTICE: This is generated code. DO NOT EDIT! BOOL Checker. Capture. Credit: : Put(Binary. Encoder &_encoder) const { _encoder. Begin. Message(CODE, NAME); _encoder. Put("captured. Checker. ID", captured. Checker. ID); _encoder. Put("captured. By", captured. By); _encoder. Put("jump. Type", jump. Type); _encoder. End. Message(CODE, NAME); return TRUE; }

Flex and Bison make writing parsers easy Other tools • ANTLR • GOLD • PLY (Python Lex & Yacc) • Boost. Spirit Flex & Bison - parser generators

JAM File syntax is described to Bison part of jam. y

From. jam to. cpp 2004 2013

TRL Turns. jam into C++ {@ define Output. Message(msg, encoders, decoders) @} // // NOTICE: This is generated code. DO NOT EDIT! // class {{ msg. struct. Name }} : public Jam. Message { public: static u 32 CRC; static u 16 CODE; static cchar *NAME; // No argument constructor: {{ msg. struct. Name }}() { {@ foreach f in msg. fields @} {@ if f. has. Default @} {{ f. name }} = {{ f. def. Value }}; {@ end if @} {@ end foreach @} } TRL to generate a message class definition See Also • CTemplate • ng. Template • Django (HTML focused) • Jinja (Python)

Fill out a dictionary and feed it to TRL {@ foreach f in msg. fields @} {@ if f. has. Default @} {{ f. name }} = {{ f. def. Value }}; {@ end if @} {@ end foreach @}

Global Feature addition using TRL TODO: Figure out how to illustrate this. Looking through history of our TRL files and jamgen would be useful. In fact, a screenshot of a diff of one of those changes would work well here.

Development Cycle • Describe the protocol • Generate serialization and dispatch • Send messages • Receive messages • Configure routing info

Create a message, fill in data, call send void Checker: : On. Captured(Checker. ID captured. By, JUMP_TYPE how) { Checker. Captured. Credit msg; msg. captured. Checker. ID = Get. ID(); msg. captured. By = captured. By; msg. jump. Type = how; Jam. ID destination = Get. Router()->Get. Credit. Manager. ID(); Get. Router()->Send(destination, &msg); }

Structs and arrays in messages message Group. Update { Group. ID group; array<. Checker. ID> checkers; }; /*** DATA START ***/ Group. ID group; vector<Checker. ID> checkers; /*** DATA STOP ***/ void Group. Service: : Send. Update(Group. ID id) { Group. Update msg; msg. group = id; msg. checkers. resize(MAX_GROUP_SIZE); //. . . }

Definitions • Message - serialized structure defined in a. jam file • Protocol - a collection of messages • Service - a module of code that implements message handlers for one or more protocols • Program - can be composed of multiple services

Message Destinations void Match. Service: : Create. Board(u 64 width, u 64 height) { Board. ID = Generate. Board(); // Send to a known, connected, service m_p. Server->Send(m_board. Server. ID, &msg); } void Match. Service: : Game. Over(u 32 game. ID, u 64 winner. ID) { msg. game. ID = game. ID; msg. winner = winner. ID(); // Send to a service type, non-specified ID m_p. Server->Send(JAM_SERVER_STATS_TRACKER, &msg); m_p. Server->Broadcast(JAM_SERVER_STATS_TRACKER, &msg); } void Checker: : Heal. Checker(Checker. ID to. Heal, u 32 amount) { Checker. Heal msg; msg. healed. By = Get. ID(); msg. amount = amount; // Send a message to a specific object m_p. Server->Send(to. Heal, &msg); }

Message routing by type Matchmaker. Add. Player add. Msg; add. Msg. player = Get. Player. ID(); add. Msg. rank = Get. Rank(); // No Jam. ID needed, send to any Matchmaker // May be queued until a Matchmaker is available m_p. Service->Send(JAM_SERVER_MATCHMAKER, &add. Msg);

Send a message and expect a response Matchmaker. Add. Player add. Msg; add. Msg. player = Get. Player. ID(); add. Msg. level = Get. Level(); // Send to any Matchmaker, Player. Added. Handler // will be called with response when complete m_p. Service->Send. Registered<Player. Added>( JAM_SERVER_MATCHMAKER, &add. Msg );

Send a message to an object void Checker. Group: : Change. Boards(u 32 new. Board) { Checker. Change. Board msg; msg. board. ID = new. Board; for(int i = 0; i < m_checkers. size(); i++) { m_p. Server->Send(m_checkers[i]->Get. ID(), &msg); } }

Each object is owned by one server class Checker { //. . . Checker. ID m_id; Jam. ID m_server. ID; Jam. ID Get. Server() { return m_server. ID; } Checker. ID Get. ID() { return m_id; } //. . . };

How messages get routed void Board. Server: : Send(Checker *p. Checker, Jam. Message *p. Message) { m_p. Jam. Server->Send(p. Checker->Get. Server(), p. Checker->Get. ID(), p. Message); }

Development Cycle • Describe the protocol • Generate serialization and dispatch • Send messages • Receive messages • Configure routing info

On receipt, look up and dispatch // static callback registered with JAM by protocol ID // called for each incoming message void Board. Server: : Checker. Dispatch(Jam. Link &link, Jam. Message *p. Message) { Checker. ID dest. ID = p. Message->Get. Destination(); Checker *p. Checker = Get. Checker. Object(dest. ID); p. Checker->Queue. Message(p. Message); switch(p. Message->Get. Protocol. CRC()) { case JAMChecker. Protocol_CRC: Jam. Checker. Protocol: : Dispatch<Checker>(p. Message, p. Checker); } }

Jam. Link &link, void Board. Server: : Checker. Dispatch( Jam. Message *p. Message) {

Generated Dispatch methods //NOTICE: This is generated code. DO NOT EDIT! template<typename HANDLER_T> static JAM_RESULT Dispatch(Jam. Message *p. Message, HANDLER_T *p. Handler) { switch(p. Message->Get. Code()) { case JAM_MSG_Checker. Heal: result = p. Handler->Checker. Heal. Handler(link, (Checker. Heal *)p. Message); break; // cases for rest of protocol's messages. . .

Generated message handler prototypes // A message handler prototype is auto-generated for each message // in the protocol. #include these declarations in the middle // of your hand constructed class. JAM_RESULT Checker. Heal. Handler(Jam. Link &link, Checker. Heal *msg); JAM_RESULT Checker. Damage. Handler(Jam. Link &link, Checker. Damage *msg); JAM_RESULT Checker. Powerup. Handler(Jam. Link &link, Checker. Powerup *msg); JAM_RESULT Checker. King. Handler(Jam. Link &link, Checker. King *msg); #include this in the middle of a class

Message handler methods JAM_RESULT Checker: : Checker. Heal. Handler(Checker. Heal *p. Message) { m_health += p. Message->amount; LOG("Checker %d was healed for %d by checker %d", Get. ID(), p. Message->amount, p. Message->healed. By); return JAM_OK; }

Send and Receive void Checker: : Heal. Checker(Checker. ID to. Heal, u 32 amount) { Checker. Heal msg; msg. healed. By = Get. ID(); msg. amount = amount; // Send a message to a specific object m_p. Server->Send(to. Heal, &msg); } JAM_RESULT Checker: : Checker. Heal. Handler(Checker. Heal *p. Message) { m_health += p. Message->amount; LOG("Checker %d was healed for %d by checker %d", Get. ID(), p. Message->amount, p. Message->healed. By); return JAM_OK; }

Development Cycle • Describe the protocol • Generate serialization and dispatch • Send messages • Receive messages • Configure routing info

Define services void Matchmaker: : Configure(Jam. Server *p. Server) { Jam. Route. Config &route. Config = p. Server->Get. Route. Config(); route. Configure. Inbound<Matchmaker. Protocol>( this, Matchmaker: : Dispatch. Message); route. Configure. Outbound<Matchmaker. Response. Protocol>(); } Configure protocols the Matchmaker service sends and receives

Route. Config maintains a protocol to handler mapping

Handlers have access to sender and other metadata about received messages JAM_RESULT Board. Server: : Add. Player. Handler(Jam. Link &link, Add. Player *msg) { LOG("Adding player %s from server %s", IDSTR(msg->player. ID), link. Describe(). c_str()); // Do stuff return JAM_OK; }

Coarse and fine-grained queueing and locking strategies are available Race Condition

Receiving via Message Queue void Matchmaker: : Configure() { // Messages received at any time are placed into a queue route. Configure. Inbound<Matchmaker. Protocol>( this, &m_message. Queue); } void Matchmaker: : Idle() { // Queue is processed in one thread at a known time p. Server->Process. Queue(&m_message. Queue, this); }

Global lock dispatching

Raw concurrent handlers

Lock Policies class Matchmaker. Lock. Policy { Matchmaker *m_owner; void Lock(Jam. Message *msg, Jam. Message. Queue **pp. Queue) { // Adding a player requires a write lock if(msg->Get. Code() == JAM_MSG_Matchmaker. Add. Player) { m_owner->Acquire. Write. Lock(); } else { m_owner->Acquire. Read. Lock(); } } void Unlock(Jam. Message *msg) { /* Same logic, release lock */ } }

Incoming messages are refcounted • Message passed to handler is a refcounted object • Possible to retain a message pointer until later • Smart pointers are available • Messages contain no pointers to any other objects • No circular references are possible

CPU And Bandwidth Efficiency

JAM is either efficient or backwards compatible 2004 - Assumed binary compatibility

Negotiation means dead-simple binary serialization most of the time

In some cases, can just memcpy it onto the wire // This message could easily be memcpy'ed onto the wire class Create. Checker : public Jam. Message { /**** DATA START ****/ u 32 checker. Type; u 32 owner; /**** DATA STOP ****/ // Code. . . };

Generated code means easy optimizations _encoder. Put("captured. Checker. ID", captured. Checker. ID); _encoder. Put("captured. By", captured. By); _encoder. Put("jump. Type", jump. Type);

Fallback to JSON Serialization • Switch to JSON serialization when binary CRC check fail • Great for programmers • Way more expensive (CPU and Bandwidth) • Never allowed on public facing protocols • Even internally it’s sometimes unreasonable

JSON { "_msg. ID": 10, "type": 6, "error": 0, "desc": { "m_id": "T 2 R 00 S 40. 00 E 14815726 P 10987 H 127. 0. 0. 1: 14001", "m_host": "127. 0. 0. 1", "m_partition. ID": 0, "m_config. ID": 0, "m_build. Num": 0, "m_type": 40, "m_sub. Type": 0 } }

Protocol Negotiation

Message overhead

JSON vs. Binary performance

Still highly successful - some network tools run on old versions frequently

Facebook's Thrift are good alternatives

For both speed AND inter-version compatibility, there are better choices

protobufs sometimes wins on bandwidth, but JAM is faster

Writing our own gives us ultimate control over everything • Automated serialization • Easy yet flexible message dispatching • High performance • Inter-version compatibility • Less tedium = more awesome

Thanks! Questions? Joe Rumsey [email protected] com Twitter: @joerumz Disclaimer: Blizzard is not really making World of Checkers
World of warcraft api
Candy crush versus
Network serialization
Goldhax. com pubg
Wow tbc system requirements
Goodrich method flood routing
Mark tinka
Continuity equation hydrology
Difference between clock routing and power routing
Performing file i/o using hdfs
Brazil serialization requirements
What is the use of serialization
Kotlinx serialization generics
Jws json serialization
Drug supply chain security act timeline
Kotlinx serialization performance
Fungsi network layer adalah …
Network slicing with segment routing
A switched wan is normally implemented as a network
Features of peer to peer network and client server network
Network centric computing and network centric content
Ap world history chapter 25 africa and the atlantic world
The changing world output and world trade picture
The changing world output and world trade picture
Ortec routing and dispatch
Interplay between routing and forwarding
Address aggregation simplifies the forwarding process in
Principles of good routing and scheduling
Rip static
Advantages of link state routing vs distance vector
Masking and data routing mechanism
Routing fabric
Hong kong university
Lab 4-1: routing concepts and protocols
Broadcast and multicast routing
Igmpv
Routing and switching protocols
Topology in computer
Network systems design using network processors
Packet switched vs circuit switched
Netography
The internet is the world's biggest
The internet is a vast computer network
Mfa small world network
Old world monkey vs new world monkey
Are oranges old world or new world
Real world vs digital world
Allegory of the cave examples in real life
Bad world tour
Shakespeare's head is an example of
Wear skins peeped through by bones figure of speech
Unit 9 english in the world looking back
Ap layer in vlsi
Wrp routing protocol
Mppc adalah
Segment routing
Rumor routing
Routing statyczny
Routing overlays
Tabel routing static
Routing operation
Path vector routing