Reflection for Tools Development Jeremy Walker Senior Tools

Reflection for Tools Development Jeremy Walker Senior Tools Programmer Ubisoft Vancouver

Table of Contents 1. The Problem • Understanding the Costs of Tools Development 2. Reflection • The Ultimate Technique for Data Driven Solutions 3. Content Framework • A Reflection-Powered Infrastructure for Tools

1. The Problem Understanding the Costs of Tools Development

Hard Coded vs. Data Driven Systems Data Driven System Hard Coded System Animation Tools Animation Build System Hard Coded Animation System Animation Serialization Animation Engine

What's the Cost of Great Workflows? Sounds Animation Scenes Tools Sound. Animation Editing Tools Scene Editing Tools Sound Build System Animation Build System Scene Build System Animation Serialization Sound Serialization Scene Serialization Sound Playback Animation. Engine Scene Rendering Engine

Great Workflows for All Types of Content Animation Sounds Scenes SHARED Textures AI Levels SHARED Input Front End Cameras Particles SHARED Models Shaders Materials HUD SHARED Curves Videos Skeletons Music Sequences State Flows SHARED SHARED Characters Props SHARED Scripts Cut Scenes SHARED

The Keep-It-Simple Approach Animation Sounds Scenes Input Front End Cameras Particles Characters Hard Coded SHARED Shaders Materials HUD Props Hard Coded SHARED Textures AI Levels Models Hard Coded SHARED SHARED Videos Skeletons Music Sequences State Flows Curves Hard Coded Hard Coded SHARED SHARED Scripts Cut Scenes SHARED

The Monolithic Engine Approach Animation Sounds Scenes Monolithic Engine Input Front End Cameras Particles Characters Materials HUD Props Videos Scripts Cut Scenes Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes Textures AI Levels Animation, Models Sounds, Scenes, Input, Front. Shaders End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes Skeletons Music Sequences State Flows Curves Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes

The Problem Low Development Cost Adaptable To Change Reusable Systems Game and Genre-Specific Needs ? Stable Tools Great Workflows

The Solution For all types of content: • Minimize the cost of developing great workflows. • Design reusable systems while accommodating gamespecific needs. • Develop stable tools that adapt to constant change during production.

The Package Release Cycle Source Code Release Candidate Release yes Programmer Modifies Code Automated Build Process OK? no Testing Process OK? no Artist Gets Package Release Artist Requests New Features

The Monolithic Package Release Problem Monolithic Package Source Code Monolithic Package Release

The Benefit of Decoupled Packages Package Source Code Package Release Package Source Code Package Release

The Lockstep Release Problem Animation Tool Source Code Animation Tool Release Compile Time Dependency Run Time Dependency Sound Tool Source Code Compile Time Dependency Game Source Code Lockstep Release Cycle Sound Tool Release Run Time Dependency Game Release

Reducing Cost and Complexity Hardcoded Engine Monolithic Engine Decoupled Systems Animation Sounds Scenes Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes Cameras Particles Characters Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes Materials HUD Props Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes Hard Coded Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes Low Cost Simple Hard to Reuse Poor Workflow Medium Cost Complex Reuse All or Nothing Great Workflow Reflection Low Cost High Very. Simple Complex Very Reusable Very Great Workflow Great

2. Reflection The Ultimate Technique for Data Driven Solutions

What is Reflection? From Wikipedia: “…Reflection is the process by which a computer program can observe and modify its own structure and behaviour. ”

Reflection is Used Everywhere

Case Study: C++ Compilation Discover Function B calls Function A C++ Compiler Validate Function Declaration Reflection Binding to Function Definition Expose C++ Compiler Implement Function A

Reflection for Language Interoperability Discover C# Code C# Compiler Validate Type Definitions Reflection Type Binding Expose Visual Basic Compiler Implement Visual Basic Code

Reflection for Scripting in Games Discover Lua Script Adapter Validate Type Definitions Reflection Type Binding Expose C++ Adapter Implement C++ Classes

Reflection for Serialization in Games Discover Serialized Objects Serializer Validate Type Definitions Reflection Type Binding Expose C++ Adapter Implement C++ Classes

Common Language Specification Type Definitions Reflection Type Binding • Type size and alignment • Allocate objects • Native types • Binary representation • Classes • Invoke constructor, get/invoke members • Enums • Get/set enum value

Reflection in Action Example: Print all fields of type “float” const Type* type = Type. Of<Simple. Vehicle>(); const char* name = type->Name(); size_t size = type->Size(); printf(“Type: %sn Size: %dn”, name, size); Instance vehicle = type->Create. Instance(); vehicle. As<Simple. Vehicle&>(). m_Max. Speed. KPH = 193. 12 f; for (int i=0; i<type->Num. Members(); i++) { const Member* member = vehicle. Get. Member(i); if (member->Is. Field()) { const Field* field = (Field*)member; if (field->Get. Type() == Type. Of<float>()) { Instance value = vehicle. Get. Field(field); printf(“Field: %sn Offset: %dn Value: %. 2 fn”, field->Name(), field->Offset(), value. As<float>()); } } } Output Type: Simple. Vehicle Size: 4 Field: Max. Speed. KPH Offset 0 Value: 193. 12

Popular Approaches to C++ Reflection • Macros • Code Parser • Type Definition Language

Reflection with Macros Example class Simple. Vehicle : public Entity { public: DECLARE_TYPE(); float m_Max. Speed. KPH; void Reset(bool use. Defaults); float Get. Max. Speed. MPH() const; void Set. Max. Speed. MPH(float max. Speed. MPH); }; //In a separate. CPP file: DEFINE_TYPE(Simple. Vehicle) BASE_CLASS(Entity) FIELD(“Max. Speed. KPH”, m_Max. Speed. KPH) METHOD(Reset) PROPERTY(Get. Max. Speed. MPH, Set. Max. Speed. MPH) DEFINE_TYPE_END() Pros • No external tools Cons • Awkward to implement • Hard to debug • Run-time discovery

Reflection with Code Parser Example /// [Class] class Simple. Vehicle : public Entity { public: /// [Field(“Max. Speed. KPH”)] float m_Max. Speed. KPH; /// [Method] void Reset(bool use. Defaults); /// [Property] float Get. Max. Speed. MPH() const; /// [Property] void Set. Max. Speed. MPH(float max. Speed. MPH); }; Pros • Easier to implement • Compile-time discovery Cons • Slow pre-build step

Reflection with Type Definition Language Example class Simple. Vehicle : Entity { float Max. Speed. KPH; void Reset(bool use. Defaults); float Max. Speed. MPH { get; set; } }; Pros • Easiest to implement • No slow pre-build Cons • Can’t reflect existing classes

Exporting Type Definitions for Game Generated Binder Code C++ Reflected Types (game-side) Reflection System (game-side) Code Parser Types. xml C++ Class Types. xml /// [Class] <type name=“Simple. Vehicle”> class Simple. Vehicle : publictype=“float”/> Entity <field name=“Max. Speed. KPH” { <method name=“Reset” returntype=“void”> public: <parameter name=“use. Defaults” type=“bool”/> /// [Field(“Max. Speed. KPH”)] </method> float m_Max. Speed. KPH; <property name=“Max. Speed. MPH” type=“float” hasget=“true” hasset=“true”/> /// [Method] </type> void Reset(bool use. Defaults); /// [Property] float Get. Max. Speed. MPH() const; /// [Property] void Set. Max. Speed. MPH(float max. Speed. MPH); };

Exporting Type Definitions for Tools C++ Reflected Types (game-side) Code Parser Types. xml Reflection System (tool-side) C#Types. xml Proxy Class <type [Proxy. Type(“Simple. Vehicle”, name=“Simple. Vehicle”> 0 x 81 c 37132)] public <field partial name=“Max. Speed. KPH” class Simple. Vehicle type=“float”/> : Entity { <method name=“Reset” returntype=“void”> public <parameter floatname=“use. Defaults” Max. Speed. KPH type=“bool”/> </method> { <property get { name=“Max. Speed. MPH” return this. Instance. Get. Field(“Max. Speed. KPH”). Get<float>(); type=“float” hasget=“true” hasset=“true”/> } </type> set { this. Instance. Get. Field(“Max. Speed. KPH”). Set<float>(value); } } } public float Max. Speed. MPH {. . . } public void Reset(bool use. Defaults) {. . . } } C# Proxy Generator

Primary Uses for Reflection in Tools Types. xml Serialization Reflection System Generated UI (Property. Grid, Tree. View, etc. ) Client-Server Remoting

Client-Server Remoting Tools (Client) Discover Generated C# Proxy Classes C# Proxy Generator Validate Type Definitions Reflection Type Binding Game (Server) Expose Remoting Client Implement Discover Remoting Server Validate Type Definitions Reflection Type Binding Expose C++ Adapter Implement C++ Game Classes

Problems and Workarounds Type definitions out of sync • • Detect type checksum mismatch Detect problems early Auto-synchronization of type information Auto-migration of data

Problems and Workarounds Tools tightly coupled to the game • • • Avoid overuse of generated proxy classes Use generated UI where possible Use polymorphic proxy classes Tools Code has proxy Base Class Derived Class A Derived Class B Derived Class C has no proxy

Problems and Workarounds Excessive memory usage • • Strip type information based on usage Auto-detect unused reflected types

Other Uses for Reflection • • • Marshalling events for multi-processor architectures Client-Server Remoting for Online Serialization of saved game data

3. Content Framework A Reflection-Powered Infrastructure for Tools

Content Framework Tool-side Game-side 3 rd Party Content Framework Asset Repository Package System Asset Explorer Build System Asset Manager Asset Previewing Reflection

(Demo Videos)

Promising Results • Fast tools development • Great workflows for all types of content • Decoupled systems with improved reusability and resilience to change

Questions?