GDC 2011 Jubok Kim ChoongHyo Kim I m

GDC 2011 | Jubok Kim, Choong-Hyo Kim I m p l e m e n ta ti o n a n d A p p l i c a ti o n o f th e Real-Time Helper-Joint System Ⓒ 2011 NEXON Corporation & dev. CAT Studio. All Rights Reserved M 2 team, Game Development Team for Project M 2 in long. CAT (The 3 rd New Development Division in NEXON Corp. ). M 2 team Director is Kim, Dong-Gun | Project M 2 is produced by Kim, Dong-Gun GT-R team, Engine Development Team for Project M 2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

Jubok Kim Technical Director of “Mabinogi 2” Worked as a game programmer for 10 years in Nexon eias@nexon. co. kr http: //twitter. com/eiaserinnys Choong-Hyo Kim Technical Art Director of “Mabinogi 2” Worked as a 3 D artist for 10 years in Nexon uc 2612@nexon. co. kr http: //twitter. com/siyoskii

Nexon? Nexon is the first Korean online game company to direct its attention to the overseas market Cartoon Rendered-MMORPG, Mabinogi A Casual MMORPG Maple. Story MMO-Action Vindictus (Mabinogi: Heroes in

Mabinogi 2 “Mabinogi 2” is the third one of the series and the official sequel to “Mabinogi. ” We are aiming very high quality bar of graphics and animation unlike its casuallooking prequel

What we’re gonna talk. 1. Real-time helper-joint system Problem definition Study about helper-joints Replication of helper-joints 2. Layering the helper-joints Concept / Authoring conventions Implementation 3. Optimization 4. Conclusion Pros/Considerations **Annotation slides

Problem Definition

Candy-wrap problem Basic bone structure + Skinning = Twisted wrist You will see problems like this even at shoulder or other complex joint Error OK… Candy Wrap

Twist bone is a solution Measure and calculate the average angle between forearm and wrist to get the rotation of the twist bone

Helper-Joint A joint which measures the movement of other joints and settle the movement of itself Many DCC tools provide features which are actually helper-joints Popular controllers in 3 DS MAX : Position Constraint Look At Constraint Orientation Constraint…

So, what’s the problem left? All we’ve got helper-joints in our DCC tools already.

Hard to modify behavior All animation asset should re-baked if you change behavior of a helper-joint It becomes more and more painful to modify behavior of the skeletal structure in the later stage of development because animation asset size is growing bigger and bigger

Procedural Motion Helper-joint does not react properly to procedural animation like ragdoll or fullbody IK http: //forum. unity 3 d. com/threads/51805 -Additive-animations-mess-up-skinning

Customization Hard to author and modify costumes if you need to bake all the animation for them. (Most of typical MMOGs have rich character customization feature) Costumes from Vindictus, Nexon

We needed Helper-Joint which does not require pre-baked animation Let’s call it “Real-Time Helper-Joint”

Expected effects with Real-Time Helper-Joint system are… Flexibility of the content creation pipeline Smaller package size to distribute Proper reaction to procedural animation Low cost for authoring various and complex costumes

Problems predicted We predicted some problems before beginning to build the system Inexperience Artists were not experienced to the concept of Helper-Joint. We did not use Helper-Joint for the previous game, and materials are very rare in Korea. Hardness of replication We did not know even if it was possible to replicate arbitrary Helper-Joint in DCC tool. There was a high chance of failing if we try to replicate too complex and unpredictable Helper-Joint. Performance Issue Even if we succeeded to replicate Helper-Joint in the game code, can we get enough performance to process many characters in the crowd scene of an MMOG?

Solving Strategy We prepared clear strategies for each predicted problems before starting to build the system for Inexperience Technical artist decided to study about human skin changes rather than muscle movement, because Helper-Joint is about skinning for Hardness of replication Technical artist and programmer reached an agreement to maintain the list of Helper-Joints of 3 DS MAX compact by choosing easy and intuitive ones for Performance Issue Programmer decided to adopt simple and rigid architecture to support multi-threading

Study about Helper-Joints

Where do we need it? We collected cases which is hard to set up skinning without Helper-Joint first Shoulder Usually hard part to set up Pelvis

Upper Am Forearm Parts requiring twist bone Thigh

Elbow Parts with caution for rubbery joint problem Ankle

Pectoral Muscles Scapula & Change of Knee Bulge of Thigh Parts affect to attractive character silhouette

Focused on female shoulder Female shoulder was the best study subject with all the reviewed cases Why shoulder? Shoulder has 3 degrees of freedom Most skinning issues get resolved if we solve the issues around shoulder

Focused on female shoulder Female shoulder was the best study subject with all the reviewed cases Why female? Expression of subtle silhouette of a female character is much more harder than tough expression of muscles of a male character

More early decisions We made some more decisions before start to set up female shoulder 3 DS MAX native features only Maintenance cost of the system might grow bigger, if we develop our own Helper-Joint system (what if there is 3 DS MAX upgrade? )

More early decisions We made some more decisions before start to set up female shoulder Focus on movement of skin only The system would be an over-engineered one if we have studied and replicated the movement of muscles

Rotation around the longitudinal axis skin Markers on my body Rotation limits le nd the deltoid musc Little rotation arou

Rotation with the arm forward le. nd the deltoid musc Little rotation arou kes Smoothe curve. The arm rotation ma

Rotation with the arm forward id. curve on the delto Changes along the

Rotation with the arm upward from the Silhouette the latissimus. n a iceps, d deltoid, tr Less deformation per arm. at the root of the up Noticeable crease of the skin surface Bulging deltoid

Basic Components Basic components to settle movement of a Helper-Joint Animation of the base framework Nodes for Measurement Controllers for intermediate calculation

No detail for the rig Will not talk about it right now. There’s No Fancy technique. Keep it simple.

Final rig Preview with 3 ds max

Demonstration V i d e o 1 This demonstration shows the our final rig with 3 DS MAX native Helper-Joints

Final list to replicate We determined the minimal list of Helper-Joints to replicate by the study Position. Constraint Look. At. Constraint Orientation. Constraint List. Controller (S/R/T) Reaction. Controller (S/R/T/Float) Expression. Controller (S/R/T/Float) Wire. Parameter we decided not to use Wire. Parameter later, because it is too slow and hard to modify Expose. TM

Replication of Helper-Joints

Position. Constraint Let’s start with the simplest Helper-Joint in the list mentioned before Simply adds and calculates the average of position targets multiplied by weight

Preparation Technical artist part Sample Scene Technical artist created a scene 2~3 bones using position constraints and export its setting as an XML file by MAXScript

Preparation Programmer part Interface Definition Programmer defined the simple interface for Helper-Joint and integrate it into the existing animation system struct Helper. Joint. Result { enum Type { Scale, Rotation, Position, }; Type type; Vector 4 result; }; class IHelper. Joint { virtual void Process(Current. Pose& pose) = 0; virtual const Helper. Joint. Result& Get. Result() const = 0; };

Pseudo-code Really easy one, huh? class Position. Constraint : public IHelper. Joint { virtual void Process(const Current. Pose& pose) { Calculate the matrix T which transforms a ‘world space coordinate’ into a ‘parent space coordinate’ Calculate the average position P of the positions of target bones multiplied by weight in the world space Transform P by T and store it } virtual const Helper. Joint. Result& Get. Result() const { return P; } };

Life isn’t that simple… 3 DS MAX calculates and stores the differences between bind pose and the weighted average of target positions in the bind pose, and add them to later calculation results if ‘Keep Initial Offset’ option is checked Bone position from Position. At. Constraint Actual bind position Position adjusted by ‘Keep initial offset’ option

Introducing preprocess step Preprocess() function has been added to IHelper. Joint interface to handle the ‘Keep Initial Offset’ option class Position. Constraint : public IHelper. Joint { virtual void Preprocess(const Bind. Pose& bind. Pose) { if ‘keep initial offset’ option is cheked { Run Process() with bind pose and store the result in P_base Calculate the differences of position between P_base and actual bind pose and store the results in Offset } } virtual void Process(const Current. Pose& pose) { Calculate the matrix T which transform a ‘world space coordinate’ into a ‘parent space coordinate’ Calculate the averaged position P of the positions of target bones multiplied by weight in the world space Transform P by T and store it if ‘keep initial offset’ option is cheked Add the preprocessed Offset to P } }

Replicate the remainder It is straightforward to replicate except non-intuitive options like ‘Keep Initial Offset’ Detailed implementation note of other Helper-Joints will be added in appendix

Position. Constraint Look. At. Constraint Orientation. Constraint List. Controller (S/R/T) Reaction. Controller (S/R/T/Float) Expression. Controller (S/R/T/Float) Wire. Parameter Expose. TM Cool, it’s all xxxxing done All replications are done, but this is too easy, and bad premonition always proved right…

Unexpected problem The order of evaluation was not a straightforward one In general animation calculation process, if you calculate the world transform in order, accurate result will be guaranteed 0 1 8 2 3 5 4 6 9 7 11 10 12 13 14

0 1 8 Look. At Constraint 2 3 5 4 Unfortunately, this simple and easy solution does not work with Helper-Joint 6 7 11 Look. At 9 Target 10 12 13 14 A Helper-Joint can refer to the bone which is not its own ancestor

A naïve solution? The order of evaluation can be calculated by traversing Helper-Joints in postorder To traverse Helper-Joints, we should figure out their dependency first /* Let’s add a function to the IHelper. Joint interface to return the list of bone indices which the Helper-Joint referred to */ class Look. At. Constraint { virtual void Get. Dependency( vector<Bone. Index>& dependency) { Add indices of look-at targets to dependency Add index of upnode to dependency } /* Other functions */ }

is not a rigid solution… We have a lot of Helper-Joints to implement, so more solid and human error proof solution is required /* What if there is more complicated Helper-Joint like an expression controller? */ /* Furthermore, how do I validate and convince the dependency list, and evaluate that order is correct? */ class Reactor. Expression. Binary { void Get. Dependancy(std: : vector<int>& }; class Reactor. Expression. ID { void Get. Dependancy(std: : vector<int>& }; class Reactor. Expression. Function { void Get. Dependancy(std: : vector<int>& }; class Reactor. Expression. Constant { void Get. Dependancy(std: : vector<int>& }; dependancies);

Concrete solution Use evaluation step itself as dependency traversal step Each implementation of Helper-Joint is modified not to access the animation pose array directly. Instead, Helper-Joint uses IHelper. Joint. Pose. Source whenever it requires pose of other bones Position. Constraint IHelper. Joint. Pose. Source Look. At. Constraint Get. World. Translation(int bone. Index) Get. World. Rotation(int bone. Index) Get. World. Scale(int bone. Index) Get. Local. Translation(int bone. Index) Get. Local. Rotation(int bone. Index) Get. Local. Scale(int bone. Index) Get. World. Transform(int bone. Index) … Orientation. Constraint Position. Reaction. Controller Rotation. Reaction. Controller Scale. Reaction. Constraint Float. Wire. Constraint

A special implementation of IHelper. Joint. Pose. Source Evaluation. Order. Builder is used when order of evaluation is calculated. Position. Constraint 9, 10 Orientation. Constraint 12 Scale. XYZ bone 11 Evaluation. Order. Builder : public IHelper. Joint. Pose. Source Position. XYZ Look. At. Constraint 24 Scale. XYZ bone 12 Assumes that bones with index less than 11 are already evaluated … 10

Position. Constraint 9, 10 Bone 9, 10 are already evaulated, so they can be referenced without problem. Orientation. Constraint 12 Scale. XYZ bone 11 public IHelper. Joint. Pose. Source Position. XYZ Look. At. Constraint 24 Scale. XYZ bone 12 Evaluation. Order. Builder : … 10

Position. Constraint 9, 10 Orientation. Constraint 12 Scale. XYZ bone 11 Evaluation. Order. Builder : public IHelper. Joint. Pose. Source Position. XYZ Look. At. Constraint 24 Scale. XYZ bone 12 Bone 12 is not traversed yet. So evaluation of bone 11 is holded, and evaluation of bone 12 begins instead. … 10

Position. Constraint 9, 10 Orientation. Constraint 12 Scale. XYZ bone 11 Evaluation. Order. Builder : public IHelper. Joint. Pose. Source Position. XYZ Look. At. Constraint 24 Scale. XYZ bone 12 If a bone is calculated by pre-baked animation, then it can be referenced anytime. Let’s assume bone 24 is settled by pre-baked animation. Bone 24 is assumed to be traversed now, and added to the traversal order list. … 10 24

Position. Constraint 9, 10 Orientation. Constraint 12 Scale. XYZ bone 11 Evaluation. Order. Builder : public IHelper. Joint. Pose. Source Position. XYZ Look. At. Constraint 24 Scale. XYZ bone 12 Bone 12 is evaluated, so add it to the traversal list. … 10 24 12

Position. Constraint 9, 10 Orientation. Constraint 12 Scale. XYZ bone 11 Evaluation. Order. Builder : public IHelper. Joint. Pose. Source Position. XYZ Look. At. Constraint 24 Scale. XYZ bone 12 Now bone 11 can reference bone 12 safely. … 10 24 12

Position. Constraint 9, 10 Orientation. Constraint 12 Scale. XYZ bone 11 Evaluation. Order. Builder : public IHelper. Joint. Pose. Source Position. XYZ Look. At. Constraint 24 Scale. XYZ bone 12 Bone 11 is evaluated now, so add it to the list too … 10 24 12 11 Bone 12 is already evaluated, so evaluate bone 13 next.

Demonstration V i d e o 2 This demonstration shows the basic implementation result. (Shoulder, Knee, reaction to IK)

Layering Helper-Joints

Customization Let’s extend the realtime Helper-Joint system to character customization MMOGs have many kinds of costumes Many of them require its own animation. And costumes can be changed at any time. Too Big to bake them out We can’t export all the costume animations. Single base animation …for all the costumes.

Layering skeletal structure We introduced the concept of layer of the skeletal structure aked Pre-b Most pre-baked animation contain poses for this layer Body Layer ked a Pre-b Face and separate pose clips Face & Hands ural d Proce dura Proce Contains Helper-Joint settings of shoulder, knee and others Contains Helper-Joint settings of costumes Helper-Joint for body Helper-Joint for costume l

Layering example Basic concept for framework layering Sets of various frameworks Hand Nude body Long Skirt Face Short Skirt Muscle Mantle Base

Layering example Basic concept for framework layering Sets of various frameworks Hand Face Mesh Long Skirt Face Short Skirt Muscle Mantle Base

Layering example Basic concept for framework layering Sets of various frameworks Hand Robe Mesh Long Skirt Face Short Skirt Muscle Mantle Base

Layering example Basic concept for framework layering Sets of various frameworks Hand Short Skirt Long Skirt Face Short Skirt Muscle Mantle Base

Authoring convention We used a naming convention to determine the layer where a bone is contained and what the bone is for _0 Base{B}#Spine 1 Skin Weight Layer Pre-baked/Runtime _ 0 Base {B} _ 0 Base 1 Face {B} : Baked + ~ 1 Hand 1 Muscle 2 Costume 2 Tool {D} : Deformable Name # Spine 1

Authoring convention A layer should not be cyclic, or bi-directional (Frm Layer 0) _Layer 0{B}#AAA (Layer 0 + Layer 1) (Animation Data) _Layer 0{B}#BBB _Layer 0{B}#AAA _Layer 0{B}#BBB _Layer 1{B}#CCC (Frm Layer 1) _Layer 0{B}#BBB _Layer 1{B}#CCC _Layer 0{B}#AAA _Layer 0{B}#BBB _Layer 1{B}#CCC

Export and import Exporting from and importing back into the DCC tool is done layer by layer AAA. max AAA. Mesh AAA. Layer 0. Framework AAA. Layer 0. Animation Pre-baked Layer “I’ll use these layers. ” AAA. Layer 1. Framework AAA. Layer 1. Helper. Joint. Setting AAA. Layer 2. Framework AAA. Layer 2. Helper. Joint. Setting

Framework template We maintained a list of typical framework settings as templates Predefined framework templates Popular fantasy costumes like skirts, and robes Delivered to the outsourcing company Low cost for modification Behavior of the procedural layers can be changed at any time with low cost Just re-export the helper-joint setting for the layer Great benefit to quality control

Implementation Animation pose player and Helper-Joint processor should be modified to handle framework layers Framework class is introduced A Framework instance maintains current stack of layers and bind poses Animation. Pose class is introduced A Animation. Pose instance contains current bone poses and flows through each animation module.

One animation player for one layer There can be multiple instances of animation players but there is one Animation. Pose instance only. Each animation player matches the bone name to the framework to get the indices to fill the final pose array in Animation. Pose instance One Helper-Joint processor for one layer There can be multiple instances of Helper-Joint processors and works similar to an animation player. Actually, the animation player and the Helper-Joint processor are implemented as a node in the general module based on the animation system Animation. Player 0 Logic 1 Base 2 Hand Advance. Processor ILayer. Player Pose SRT (Layer 0) SRT SRT Bind pose SRT (Layer 1) SRT Helper. Joint. Proc Helper. Joint. Layer. Proc SRT Bind Pose SRT (Layer 1) SRT SRT Bind pose SRT (Layer 3) Bind pose SRT

Demonstration V i d e o 3 This demonstration shows several costumes which have their own Helper-Joint setting

Optimization

Unexpected problem again Not like that we expected, multi-threading was an easy part Multithreading was an easy part Data parallel approach and Intel TBB did all the work Thanks, Intel! World transform calculation Performance loss on scale calculation

World transform calculation Usually, multiplying local transform with parent’s world transform occurs once for a bone in a frame When the calculation result of a Helper-Joint is applied… The Helper-Joint evaluation process updates local poses of bones almost randomly To make things worse, implementation of Helper. Joints require world transform of other bones in many cases then all world transform of entire sub-tree are invalidated

Typical character framework contains 100~200 bones So performance hit will be drastic if you re-calculate world transform of all bones whenever world transform of a bone is referenced

If a world transform for a bone is calculated, we would set the dirty mask for the bone If local bone pose of a bone is updated, we would reset the dirty mask of the bone and its descendants 0 0 234567 01 34 012 3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1 8 2 5 4 6 0 9 10 11 12 13 14 9 7 10 12 11 13 08 13 14 14 0 8 12 Local pose of bone 8 is updated, then reset the dirty mask of the bone and its descendants

Dirty mask This performance problem can be simply solved if each entry in the world transform array has dirty mask 0 0 234567 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Ancestor of bone 8 is 0 only 1 8 0 9 10 11 12 13 14 All bones have list of their own ancestors 08 and descendants. 12 13 14 These lists are maintained by 13 14 0 8 12 Framework instance Decendants of bone 8 is 9~14 01 34 012 3 2 5 4 6 9 7 10 11 All entries in the world transform array have dirty mask

If world transform of a bone is referred when its dirty mask is reset, we would check and calculate world transforms of its ancestors and the bone itself It can be done recursively without list of ancestors and descendants, but there is performance hit by function call in that case 0 0 234567 01 34 012 3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1 8 2 5 4 6 0 9 10 11 12 13 14 9 7 10 12 11 13 08 13 14 14 When we need world transform of bone 12, then update ancestors of bone 12 and bone 12 itself only 0 8 12

Improved, but not enough The result of performance measurement showed that much more time consumed than expected time from the complexity of Helper-Joint calculation code L 2 cache miss was the cause Fine measurement revealed that L 2 cache miss is the cause Helper-Joint is evaluated in random order, and references to world transform of other bones occur in random and sporadic manner

Access should be gathered Accesses to memory should be predicted and gathered Calculating the order of reference The order of referencing to world transform is calculated in similar way to the way order of evaluation of Helper-Joint is calculated Ready world transforms in advance Calculate world transform in the order before evaluating a Helper-Joint so the world transform can be loaded in L 2 cache Prefetch worked well many L 2 cache hot spots have been removed by this approach by combination with some prefetch instructions

Performance loss on scale Scale transform is just one multiplication step and does not impact the performance in usual animation system, but again, it is a different story from the real-time Helper-Joint system Many Helper-Joint requires inverse matrix of world transform of arbitrary bone Calculate inverse transform of a matrix is expensive. Getting SRT from a matrix is expensive too

But non-trivial scale is rare If scale is trivial, inverse transform calculation becomes getting transpose Getting SRT from a matrix becomes much simpler too -1 R = T R SRT(M) = { (1, 1, 1), Quaternion. From. Matrix(M), (M[3][1], M[3][2], M[3][3] }

Scale mask which is similar to world transform dirty mask introduced into Animation. Pose and array of world transform We would set scale mask for the bone if local pose of a bone contains non-trivial scale Getting SRT from a world transform is replaced by matrix into quaternion conversion if scale mask is not set. Matrix inversion is replaced by matrix transpose if scale mask is not set.

Result is good Performance gain comes from the fact that animation containing non-trivial scale is very rare Further optimization is possible for the cases with non-trivial scale. For example, code calculating x component transformed by inverse of parent’s world transform can be reduced into one line.

Performance chart Here goes actual performance chart for current implementation Helper. Joint Total Clocks No. of Calls Time per Call (ms) Avg count in one char Time per one char (ms)

Conclusion

pros. Animation asset maintenance became easier Polishing the behaviors of Helper-Joint became easier Authoring high quality costumes became easier Variance in monsters can be introduced easily with additional Helper-Joint layer Package size to distribute got reduced

cons. Riggers should get more familiar to the concept of realtime Helper-Joint system Relation between costume and Helper-Joint layer introduces another complexity in asset definition management Performance hit is not ignorable

Considerations Do enough research before you begin to implement. Avoid over design Make the best use of native features of DCC tools Maintenance will be hard if there are too many in-house tools Runtime Helper-Joint system has a very different runtime execution path Integrating it into the existing system needs careful consideration

Considerations Set a standard of expression and performance which artist and programmer both can reach an agreement

Jubok Kim eias@nexon. co. kr | twitter. com/eiaserinnys (Korean) Choong-Hyo Kim uc 2612@nexon. co. kr | twitter. com/siyoskii (Korean) Q&A

Thank you

ADDITIONAL NOTE Replicating 3 DS MAX Native Controllers L ook A t. C ons tra int | O rie nta tion. C ons tra int | R e a ction. C ontrolle r | E x pre s s ion. C ontrolle r&F loa t. W ire | X Y Z & L i s t C o n t r o l l e r s Ⓒ 2011 NEXON Corporation & dev. CAT Studio. All Rights Reserved M 2 team, Game Development Team for Project M 2 in long. CAT (The 3 rd New Development Division in NEXON Corp. ). M 2 team Director is Kim, Dong-Gun | Project M 2 is produced by Kim, Dong-Gun GT-R team, Engine Development Team for Project M 2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

Make it look at targets Look. At. Constraint first calculates the weighted average of look-at targets, then makes the constrainted bone to point the position ard one 02 w p u e Th to B rd n o i t c e dir d as upwa is use ion of the direct bone Up No de Point th e avrage p weighted osition o Bone 01 f a and the nd Bone 02, ratio is 5 0: 50 t k. A Loo

Relatively easy to replicate Look. At. Constraint works in a very clear and intuitive manner, and the first version of replication is also simple void Look. At. Constraint: : Process(/* Arguments omitted */) { Calculate the transform T doing [World space]→[Parent space] Tranform the positon of constrainted bone into its parental space with T Calculated the weighted average position of targets P Transform the P with T, and calculate the forward vector F Get the upward vector U by transform the upward target with T Calculate the result matrix from F and U, and convert it into quaternion }

Prepare the basis to look at Look. At. Constraint acually works in the world space, but results in local space It is favorable to do the calcuation with the inverse of parent’s world transform void Look. At. Constraint: : Process(/* Arguments omitted */) { Calculate the transform T doing [World space]→[Parent space] Tranform the positon of constrainted bone into its parental space with T Calculated the weighted average position of targets P Transform the P with T, and calculate the forward vector F Get the upward vector U by transform the upward target with T Calculate the result matrix from F and U, and convert it into quaternion }

Finding the forward vector Calculate the weighted average position of targets, and transform it with the basis transform T to find the forward vector void Look. At. Constraint: : Process(/* Arguments omitted */) { Calculate the transform T doing [World space]→[Parent space] Tranform the positon of constrainted bone into its parental space with T Calculated the weighted average position of targets P Transform the P with T, and calculate the forward vector F Get the upward vector U by transform the upward target with T Calculate the result matrix from F and U, and convert it into quaternion }

Finding the upward vector Simple and straight? void Look. At. Constraint: : Process(/* Arguments omitted */) { Calculate the transform T doing [World space]→[Parent space] Tranform the positon of constrainted bone into its parental space with T Calculated the weighted average position of targets P Transform the P with T, and calculate the forward vector F Get the upward vector U by transform the upward target with T Calculate the result matrix from F and U, and convert it into quaternion }

…takes a lot of care There are so many options for upnode control If the “world chec an identity transf kbox” is checked, the upnode targeorm is used as t Control” is e d o n p “U f o e lu va e If th bone points the d te in ra st n co e th , t” “Look. A upnode one axis of , t” en m n lig A s xi “A is If the value ward vector p u e th as n ke ta is e d upno The “Source Axis”XYZ de temines which axis will be considered upward, an Upnode Axis” determines d the “aligned to as the upward vector whewhich axis will be taken control” is “Axis Alignmen n the value of “Upnode t”

Finding the result rotation We’ve get the forward and upward vectors, so cross them to get the result rotation Does this concludes the replication of Look. At. Constraint? void Look. At. Constraint: : Process(/* Arguments omitted */) { Calculate the transform T doing [World space]→[Parent space] Tranform the positon of constrainted bone into its parental space with T Calculated the weighted average position of targets P Transform the P with T, and calculate the forward vector F Get the upward vector U by transform the upward target with T Calculate the result matrix from F and U, and convert it into quaternion }

Some options left in rollout Wow, the “Keep Initial Offset” checkbox AGAIN!

Preserving the difference The “Keep Initial Offset” option of Look. At. Constraint works similar to the same option of Position. Constaint irection The actual d of ed bone t in a r t s n o c the pose in the bind The result directio the constrainted n of by the Look. At. Conbone in the bind pose straint The bone which is constrainted by Look. At. Constraint The intermediate direction calculated by Look. At. Constraint only The final d which is thirection the interm e sum of and the difediate result preserved ference “Keep Initiaby l Offset” option

…concludes the replication The final replication which calculates and preserves the difference by “Keep Initial Offset” void Look. At. Constraint: : Preprocess() { Preserve the difference of rotation between the bind pose and the result with Process() only when “Keep Initial Offset” is checked } void Look. At. Constraint: : Process(/* Arguments Omitted */) { /* Ommitted */ Add the preserved rotation O to F when the “Keep Initial Offset” is checked Calculate the result matrix from F and U, and convert it into quaternion }

ADDITIONAL NOTE Replicating 3 DS MAX Native Controllers L ook A t. C ons tra int | O rie nta tion. C ons tra int | R e a ction. C ontrolle r | E x pre s s ion. C ontrolle r&F loa t. W ire | X Y Z & L i s t C o n t r o l l e r s Ⓒ 2011 NEXON Corporation & dev. CAT Studio. All Rights Reserved M 2 team, Game Development Team for Project M 2 in long. CAT (The 3 rd New Development Division in NEXON Corp. ). M 2 team Director is Kim, Dong-Gun | Project M 2 is produced by Kim, Dong-Gun GT-R team, Engine Development Team for Project M 2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

Takes the average rotation Orientation. Constraint calculates the weighted average of rotations of targets The con strai takes thnted bone rotation e average of Bon and Bon e 01 e 02

The simplest one ever? It even looks like that it is simpler than Position. Constraint void Orientation. Constraint: : Process(/* Omitted */) { Get local rotations of targets Calculate the weighted average of the rotations }

No, it is not a simple one It has the “Keep initial offset” checkbox too, and a very complex option which controls blend method void Orientation. Constraint: : Preprocess() { Preserve the difference of rotation D between the bind pose and the result with Process() only when “Keep Initial Offset” is checked } void Orientation. Constraint: : Process(/* Omitted */) { if (“Transform rule” is “Local to Local”) Preserve the local rotations of targets else // “World to World” Transform the rotation of targets to the parental space and preserve them Calculate the weighted average of preserved rotations Add preserved difference D to the average when “Keep initial offset” is checked Preserved the result }

Usual “Keep Initial Offset” This option works in the same manner to Look. At. Constraint void Orientation. Constraint: : Preprocess() { Preserve the difference of rotation D between the bind pose and the result with Process() only when “Keep Initial Offset” is checked } void Orientation. Constraint: : Process(/* Omitted */) { if (“Transform rule” is “Local to Local”) Preserve the local rotations of targets else // “World to World” Transform the rotation of targets to the parental space and preserve them Calculate the weighted average of preserved rotations Add preserved difference D to the average when “Keep initial offset” is checked Preserved the result }

Strange “Transform Rules” This option betrays the usual concept of animation blending void Orientation. Constraint: : Preprocess() { Preserve the difference of rotation D between the bind pose and the result with Process() only when “Keep Initial Offset” is checked } void Orientation. Constraint: : Process(/* Omitted */) { if (“Transform rule” is “Local to Local”) Preserve the local rotations of targets else // “World to World” Transform the rotation of targets to the parental space and preserve them Calculate the weighted average of preserved rotations Add preserved difference D to the average when “Keep initial offset” is checked Preserved the result }

Blending what you “see” This option is needed because target bones generally have different parents, so blended result can be very different to the result which an artist expects 15 degree rotated (30 degree in world) Child Parent A bone which is constrainted by Orientation. Constraint with “World→World” transform rule “Local→Local” case (15 + 15) / 2 = 15 World→World case (15 + 30) / 2 = 22. 5 degree 15 degree rotated (15 degree in world)

ADDITIONAL NOTE Replicating 3 DS MAX Native Controllers L ook A t. C ons tra int | O rie nta tion. C ons tra int | R e a ction. C ontrolle r | E x pre s s ion. C ontrolle r&F loa t. W ire | X Y Z & L i s t C o n t r o l l e r s Ⓒ 2011 NEXON Corporation & dev. CAT Studio. All Rights Reserved M 2 team, Game Development Team for Project M 2 in long. CAT (The 3 rd New Development Division in NEXON Corp. ). M 2 team Director is Kim, Dong-Gun | Project M 2 is produced by Kim, Dong-Gun GT-R team, Engine Development Team for Project M 2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

Input-Graph-Output All *Reaction. Managers calculate the output value through the plotted graph When the local Z rotation in e u le r angle o +1 Muscle{D}#R_K nee_B_Dummy isf -0. 002 degree, the Z rotation lue of _1 Muscle{D}#R_Legva 1_Bulge is 0. 000 This UI is VEEEEEEERY unintuitive DO SOMETHING, AUTODESK!

Too many input types! You need to standarize the way to retrieve all the possible inputs Local Rotation X/Y/Z World Rotation X/Y/Z Local Position X/Y/Z World Position X/Y/Z Distance

Abstract the input at first The main function of the *Reaction. Manager is the “float →Graph →result” transformation Distance. Evaluator Position. Evaluator Rotation. Evaluator

Position. Evaluator This class handles input types “Local Position X/Y/Z” and “World Position X/Y/Z” const float Position. Evaluator: : Process(/* Omitted */) { if (demands the position in local space) if (relative to the parent bone) return (local translation)[channel] else if (relative to a reference bone) transform into the space of the reference bone return (transformed translation)[channel] else // reference is undefined return (world translation)[channel] else return (world translation)[channel] }

Relative to a reference? Relative to the parent bone, and world space is very intuitive But the correct replication requires translation relative to the reference bone const float Position. Evaluator: : Process(/* Omitted */) { if (demands the position in local space) if (relative to the parent bone) return (local translation)[channel] else if (relative to a reference bone) transform into the space of the reference bone return (transformed translation)[channel] else // reference is undefined return (world translation)[channel] else return (world translation)[channel] }

…comes from helper-bone Helper-bone rollout have many options which are hard to implement You should agree upon the options which would be supported by the replication

Rotation. Evaluator This class handles input types “Local Rotation X/Y/Z” and “World Rotation X/Y/Z” const float Rotation. Evaluator: : Process(/* Omitted */) { if (demands the rotation in local space) if (relative to the parent bone) return (local rotation)[channel] else if (relative to a reference bone) transform into the space of the reference bone return (transformed rotation)[channel] else // reference is undefined, then returns in world space return (world rotation)[channel] else return (world rotation)[channel] }

Quaternion→Euler. XYZ It can be derived from a conversion of a quaternion to a matrix and decomposition of a matrix to euler angles

Distance. Evaluator This handles input type “Distance” const float Distance. Evaluator: : Process(/* Omitted */) { if (the reference bone is valid) returns the distance between the base bone and the reference bone else return the distance between the base bone and the world origin }

Graph lookup is easy It can be implemented in the same way as a key frame animation lookup routine const Reactor. Result Reaction. Controller. Base: : Process(/* Omitted */) { Find the largest master/slave index whose master value is not larger than the given master value in the sorted master/slave table Interpolate the reation and the next to it returns the result }

ADDITIONAL NOTE Replicating 3 DS MAX Native Controllers L ook A t. C ons tra int | O rie nta tion. C ons tra int | R e a ction. C ontrolle r | E x pre s s ion. C ontrolle r&F loa t. W ire | X Y Z & L i s t C o n t r o l l e r s Ⓒ 2011 NEXON Corporation & dev. CAT Studio. All Rights Reserved M 2 team, Game Development Team for Project M 2 in long. CAT (The 3 rd New Development Division in NEXON Corp. ). M 2 team Director is Kim, Dong-Gun | Project M 2 is produced by Kim, Dong-Gun GT-R team, Engine Development Team for Project M 2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

Uses a raw expression Both Float. Wire and *Expression. Control need parsing an expression

Replication is simple …except you need to implement an expression parser Interpreting the given expression every frame would hurt runtime performance, so you should build a parser for it But explaining how to build a parser is beyond the scope, so I will skip this part

Reuses the implementations The implementations of ID of the expression are almost same as handlers of input types of *Reaction. Manager Rotation. Evaluator, Position. Evaluator can be used in these controllers again

ADDITIONAL NOTE Replicating 3 DS MAX Native Controllers L ook A t. C ons tra int | O rie nta tion. C ons tra int | R e a ction. C ontrolle r | E x pre s s ion. C ontrolle r&F loa t. W ire | X Y Z & L i s t C o n t r o l l e r s Ⓒ 2011 NEXON Corporation & dev. CAT Studio. All Rights Reserved M 2 team, Game Development Team for Project M 2 in long. CAT (The 3 rd New Development Division in NEXON Corp. ). M 2 team Director is Kim, Dong-Gun | Project M 2 is produced by Kim, Dong-Gun GT-R team, Engine Development Team for Project M 2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

Group of float controllers We replicated 3 types of XYZ controllers - Position. XYZ, Euler. XYZ, and Scale. XYZ nel n a h c Z Y X h c a E nt e d n e p e d in n a can have float controller

Need an explaination? In special case, it performs radian to degree conversion void Euler. XYZ: : Process(/* Omitted */) { if (is it dynamically changing? ) preserve the default value foreach (controllers associated with X/Y/Z channel) result[channel] = calculate the float controller if (the controller is Float. Reaction. Control) convert result[channel] to radian convert the result euler angles into a quaternion else preserve the default value }

Group of controllers We replicated 3 types of lists – Position. List, Rotation. List, and Scale. List

Need an explaination? (2) In special case, the calculation process changes! void Rotation. List: : Process(/* Omitted */) only Lo Actually, and Or ok. At. Const { ientatio raint n C o ns require foreach (controllers in the list) s this traint [current] = result of the controller if (controller is a *Constraint) [result] = Slerp([result], [current], [weight]) else [result] = [result] * [current] }

Jubok Kim eias@nexon. co. kr | twitter. com/eiaserinnys (Korean) Choong-Hyo Kim uc 2612@nexon. co. kr | twitter. com/siyoskii Eo. D (Korean)