Adaptation dynamique de mouvements humains Dynamic adaptation of

Adaptation dynamique de mouvements humains Dynamic adaptation of human motions Ludovic HOYET Directeur de thèse : Franck MULTON Co-directeur de thèse : Taku KOMURA

Introduction Increase of utilization of Virtual Humans in many applications • – Video games, Virtual Reality, digital factories, movies, etc – Key-point: naturalness of the virtual motions and behaviors – Interactive applications • Kinematic constraints: positions and velocities • Dynamic constraints: accelerations and inertia • Other constraints: styles, disabilities, emotions, etc Ø Adaptations, reactivity, without knowledge of the future 07/10/2020 Soutenance de Doctorat 2

Introduction Naturalness of the animations • – One constraint many solutions • How to select a natural solution? • Is it correctly perceived by users? – Dynamics plays an important role: handles masses, inertia, accelerations, etc Motivations • – Impossible to obtain all the possible human motions Ø Adaptations to external perturbations – What are the main dynamic characteristics of human motion • Humans are used to motions: do we perceive dynamic parameters? User sensitivity? – Dynamic simulation costly • To what extend can we simplify physics? Believability vs. dynamic correctness 07/10/2020 Soutenance de Doctorat 3

Outline Understanding relevant biomechanical parameters Natural adaptation to dynamic constraints User perception of dynamic parameters 07/10/2020 Soutenance de Doctorat 4

Kinematic animation Background Modeling dynamics Perceptual analysis Motion adaptation Motion capture • – Capture real actor’s motions using markers and sensors – Retails the subtle details of human motion – Impossible to capture all different situations Necessity to adapt/modify captured motions to new situations • – Inverse Kinematics: find the angular configuration q satisfying the wanted constraint X – Displacement maps [Witkin 95, Gleicher 97]: prevent discontinuities by adding a smooth variation Δ 07/10/2020 Soutenance de Doctorat 5

Kinematic animation Background Modeling dynamics Perceptual analysis Motion adaptation Motion as simultaneous or sequential complex tasks • – Motion blending: complex sequences = blending of other motions – Motion graphs: sequentially reorder poses of captured motions [Kovar 02, Lee 02] Motion 1 Motion 2 Motion 3 Limitations • – Do not take physical laws into account – Necessary to respect physical constraints [Safonova 05] 07/10/2020 Soutenance de Doctorat 6

Dynamic animation Background Modeling dynamics Perceptual analysis Motion adaptation Ensure dynamic laws • – Newton-Euler formalism – Lagrange formalism (based on the virtual work principle) Two main families • – Motion synthesis: generates dynamically correct motions from dynamic equations – Motion adaptation: takes new dynamic constraints that were not present in the original motion into account 07/10/2020 Soutenance de Doctorat 7

Dynamic animation Background Modeling dynamics Perceptual analysis Motion adaptation Motion synthesis: Dynamic controllers [Hodgins 95, Machietto 09, Coros 10] • External forces Joint torques Mechanical model Controller Knowledge Constraints [Coros 10] Limitations • – – Design of the controller Parameters to tune Naturalness of the resulting motion Computation time: <2 characters in real-time in [Coros 10] 07/10/2020 Soutenance de Doctorat 8

Dynamic animation Background Modeling dynamics Perceptual analysis Motion adaptation Motion synthesis: spacetime constraints [Witkin 88, Liu 94, Liu 02] • – Compute the trajectories q(t) of a motion for the whole sequence Limitations • – – Computation time of the optimization Parameters to tune Naturalness of the resulting motion Local minimum [Liu 02] 07/10/2020 Soutenance de Doctorat 9

Dynamic animation Background Modeling dynamics Perceptual analysis Motion adaptation Dynamic modification of existing motions: motion databases [Zordan 05, Arikan 05, Komura 05] • – Reaction to short perturbations using captured reactions stored in a database [Zordan 05] [Komura 05] Limitations • – Naturalness and computation time depend on the size of the database • Search time [Komura 05]: 1 ms (10 motions) to 7. 5 ms (70 motions) ~2 characters at 30 Hz [Arikan 05] – Compromise between naturalness and computation time 07/10/2020 Soutenance de Doctorat 10

Dynamic animation Background Modeling dynamics Perceptual analysis Motion adaptation Dynamic modification of existing motions: dynamic adaptation • – Adaptation to dynamic criteria [Tak 00, Shin 03] • Correct poses to respect dynamic equations – Dynamic filters using a Kalman filter [Yamane 00, Pollard 01, Tak 05] • Physical model predicts the next correct pose • Predicted pose is corrected using the original motion Limitations • – Non-trivial parameters to tune – Kalman filter: simple predictor (computation time) vs. complex predictor (correctness) – Compromise between naturalness and computation time • 28 -DOF skeleton model, ~80 ms/frame [Yamane 03, Tak 05] 07/10/2020 Soutenance de Doctorat 11

Partial conclusion Background Modeling dynamics Perceptual analysis Motion adaptation Motion synthesis • – difficult to design, computation time, naturalness? Motion capture adaptations • – Retails the style of the actor’s motions Framework • – Dynamic adaptation to continuous external perturbations – Real-time (on-the-fly user-defined constraints) – Adaptation based on biomechanical knowledge and user perception 07/10/2020 Soutenance de Doctorat 12

Background Modeling dynamics Perceptual analysis Motion adaptation Outline Understanding relevant biomechanical parameters Natural adaptation to dynamic constraints User perception of dynamic parameters 07/10/2020 Soutenance de Doctorat 13

Background Modeling dynamics Perceptual analysis Motion adaptation Base of support Small surface of contact with the environment / body surface • Convex hull of the contact surfaces (on a flat floor) • – In computer animation: approximated using a polygon Base of Support = where contact forces Rc apply • – Co. P: Center of Pressure • • • Linked to forces exerted by contact Rc Momentum of Rc at point O: Point where the resultant momentum of the pressure forces is zero Only exist in the base of support Used in stabilometry to quantify balance 07/10/2020 Soutenance de Doctorat 14

Background Modeling dynamics Perceptual analysis Motion adaptation Dynamic balance criteria Various criteria exist in the literature • – Based on dynamic equations and the level of dynamic – Linked to the base of support – Based on a model of the virtual human [Kajita 09] Pendulum model 07/10/2020 System of particles Soutenance de Doctorat Articulated rigid body 15

Dynamic balance criteria Background Modeling dynamics Perceptual analysis Motion adaptation Quasi-static: negligible accelerations • – GCOM: Projection of the COM G on the ground [Dyson 77, Winter 95] • GCOM ϵ Bo. S Quasi-static balance Low dynamic: non-negligible accelerations • – Xco. M: Extrapolated center of mass [Hof 05] • Based on an inverted pendulum model and COM velocity • Gravity (g) and inverted pendulum length (l) 07/10/2020 Soutenance de Doctorat 16

Dynamic balance criteria Background Modeling dynamics Perceptual analysis Motion adaptation Highly dynamic motions: non-negligible accelerations + inertia • – ZMP: Zero Moment Point [Vukobratovic 69] • Linked to forces transmitted without contact Rg (accelerations and inertia) • Momentum of Rg at point O: with • Theoretically equivalent to Co. P [Goswami 99] Ø Should not go outside the Bo. S • Co. P/ZMP Equivalence 07/10/2020 Soutenance de Doctorat 17

Background Modeling dynamics Perceptual analysis Motion adaptation Dynamic balance criteria Highly dynamic motions: non-negligible accelerations + inertias • – FRI: Foot Rotation Indicator [Goswami 99] • Measures the postural instability of a humanoid through its foot rotation – ZRAM: Zero Rate of Angular Momentum [Goswami 04] • Point where the GRF should act to produce a null angular momentum derivative [Goswami 04] 07/10/2020 Soutenance de Doctorat 18

Background Modeling dynamics Perceptual analysis Motion adaptation Dynamic balance criteria Criterion Bipedal Rot. AM Floor Computation Based on Level range GCOM all Good Positions Quasi-static Xco. M all Good Velocities Low dynamics ZMP pendulum flat Good Accelerations Low dynamics ZMP particles flat Medium Accelerations High dynamics ZMP rigid body flat Heavy Accelerations High dynamics FRI flat Heavy Accelerations Low dynamics ZRAM all Good Accelerations Low dynamics Bipedal: both single and double support phases Rot. AM: computed using rotational angular momentum, i. e. inertias Floor: floor restriction 07/10/2020 Soutenance de Doctorat 19

Background Modeling dynamics Perceptual analysis Motion adaptation Methods & Setup Evaluation of the performance of 7 criteria • – To measure the unbalance state of characters – On motions with different levels of dynamics Reaching Quasi-static Oscillations Low dynamic Walking Turning Dynamic (90° and 180°) oscillations One foot support Spins Jog & jump Highly dynamic – Hypothesis: unbalance = change of the contact status (except for locomotion) Setup: 13 naive participants – 12 Vicon MX Cameras and a synchronized AMTI OR 6 forceplate • 07/10/2020 Soutenance de Doctorat 20

Background Modeling dynamics Perceptual analysis Motion adaptation Procedure Evaluate the performance of the following criteria: GCOM, Xco. M and ZMP • – ZRAM: equivalent to ZMP with pendulum model on a flat floor – FRI: distance FRI/ZMP is not significant in human locomotion [Popovic 05] • For all our motions: distance FRI/ZMP = 0. 3 ± 0. 6 mm (6% of the frames > 1 mm) – ZMP computed with three models • ZMP pendulum (ZMP pend. ) • ZMP particles (ZMP part. ) • ZMP articulated rigid body (ZMP arti. ) 07/10/2020 Soutenance de Doctorat 21

Background Modeling dynamics Perceptual analysis Motion adaptation Procedure Study the distance of the criteria to the base of support (Bo. S) • – Based on the Co. P location in the Bo. S – Measure • False unbalanced states (false alarms) – Criterion detects a fall while the subject keeps his balance • Failures of detecting unbalanced situations – The subject has to recover balance (make a step) but the criterion do not measure an unbalance 07/10/2020 Soutenance de Doctorat 22

Background Modeling dynamics Perceptual analysis Motion adaptation Results: quasi-static motions Reaching = Reaching a target + step when falling • – GCOM outside the Bo. S = loss of balance GCOM Xco. M ZMP pend. ZMP part. ZMP arti. Safe area (42%) - 0% 0% In-between area (54%) - 3. 5% 0. 9% 0. 4% 0. 7% Dangerous area (4%) - 15. 4% 8. 2% 9. 5% 11. 5% Percentage of the total number of frames where the Co. P lies in the corresponding area 07/10/2020 Percentage of frames of the corresponding area (here 8. 2% of 4%) where the criterion lies outside the base of support Soutenance de Doctorat 23

Results: low dynamic motions Oscillations • Walking Background Modeling dynamics Perceptual analysis Motion adaptation Turning Oscillations GCOM Xco. M ZMP pend. ZMP part. ZMP arti. Safe area (25%) 0% <0. 1% 0% 0% 0% In-between area (53%) 0% 0. 2% 0. 6% 0. 2% 0. 7% Dangerous area (22%) 2. 6% 5. 4% 15. 6% 19. 9% 24. 5% Turning GCOM Xco. M ZMP pend. ZMP part. ZMP arti. 99% 97% 34% 13% 14% In-between area (57%) ~100% 99% 56% 39% 45% Dangerous area (18%) ~100% 99% 59% 43% 51% Safe area (25%) 07/10/2020 Soutenance de Doctorat 24

Background Modeling dynamics Perceptual analysis Motion adaptation Results: highly dynamic motions Dyn. Oscill. One foot support Spins Jog & jump GCOM Xco. M ZMP pend. ZMP part. ZMP arti. Safe area (48%) 1. 6% 15. 6% 44. 4% 9. 9% 7. 2% In-between area (32%) 16. 5% 35. 8% 59. 6% 33% 32. 5% Dangerous area (20%) 41. 9% 51. 5% 65. 7% 63. 0% 66. 7% Spins GCOM Xco. M ZMP pend. ZMP part. ZMP arti. Safe area (14%) 42% 72. 4% 41. 9% 48. 1% 63. 8% In-between area (65%) 72% 74. 9% 72. 3% 84. 3% 88. 2% Dangerous area (21%) 73. 1% 84. 3% 83. 1% 93. 8% Highly dynamic oscillations results are similar to low dynamic oscillations ones 07/10/2020 Soutenance de Doctorat 25

Background Modeling dynamics Perceptual analysis Motion adaptation Results: highly dynamic motions Case of a jumping motion (34 frames, 0. 23 s). • ZMPs 07/10/2020 Co. P Soutenance de Doctorat 26

Co. P/ZMP comparison Background Modeling dynamics Perceptual analysis Motion adaptation ZMP: theoretically equivalent to Co. P • – Computed ZMP =? Measured Co. P – Error depending on the human model? Average error to Co. P (m) Quasi-static 07/10/2020 Low dynamic Highly dynamic Soutenance de Doctorat 27

Partial conclusion Background Modeling dynamics Perceptual analysis Motion adaptation Modeling dynamic balance • – None of the tested criteria succeeded in accurately detecting balanced and unbalanced situations – Xco. M seems to be a promising criterion even if its performance do not overcome the other criteria • Based on velocities extension to high dynamic motions? – Computed ZMP highly differs from measured Co. P for highly dynamic motions • Torques applied on the feet? • Influence of the model influence of a more accurate anthropometric model? – Difficulties to define what is balance • When is balance lost? • Need further experiments, specific protocols and multidisciplinary approaches 07/10/2020 Soutenance de Doctorat 28

Background Modeling dynamics Perceptual analysis Motion adaptation Outline Understanding relevant biomechanical parameters Modeling dynamics in human motion: measure of balance Natural adaptation to dynamic constraints User perception of dynamic parameters 07/10/2020 Soutenance de Doctorat 29

Perception of human motion Background Modeling dynamics Perceptual analysis Motion adaptation Few works focus on the user perception • – Global perception through point light displays [Johanson 73] – Effect of high graphical quality of characters • Users are more sensitive to errors [Hodgins 98, Reitsma 08] • Help to discriminate biological and non-biological motions [Chaminade 07] Metrics to evaluate the fidelity of motions • – Reinforcement learning & Predictors [Arikan 05, Ren 05] – Physical correctness of interpolated motions [Safonova 05] – Metrics for motion graphs [Matsunaga 07] 07/10/2020 Soutenance de Doctorat 30

Perception of human motion Background Modeling dynamics Perceptual analysis Motion adaptation Naturalness of simulated motions: user sensitivity to • – – – Ø Collision between objects [O’Sullivan 03, Reitsma 09] Visual tolerance to errors [Yeh 09] Errors in ballistic human motions [Reitsma 03, Majkowska 07] Do users perceive dynamic properties? Runeson [Runeson 81] • – Users evaluated the mass of a box lifted and carried by an actor represented normally or with bright patches – Results: mass of the box is well specified in the pattern of the motion • Mass range: 4, 10, 16, 22, 28 kg • Linear relation between estimated and real mass (0. 75 slope) 07/10/2020 Soutenance de Doctorat 31

Background Modeling dynamics Perceptual analysis Motion adaptation Overview Perception of minimal noticeable difference between lifted masses • – Mass scale : 2, 3, 4, 5, 6, 7, 8, 9 and 10 kg. Problems • – Biomechanical study • kinematic parameter Distinguish a difference of mass? Accuracy? – Perceptual study • Users’ perception of videos compared to kinematic parameters’ performance • Users’ perception of Computer animation sequences compared to users’ perception of videos 07/10/2020 Soutenance de Doctorat 32

Background Modeling dynamics Perceptual analysis Motion adaptation Biomechanical Study Methods & Setup • – An actor performed lifting motions from 2 kg to 10 kg – Motions captured using 12 Vicon MX 40 cameras – Automatic selection of lifting sequences Procedure • – Study if significant differences can be found in joint kinematics using the different criteria – For each mass mi: reference = trajectory closest to the average = refmi – Significant difference between trajectories of mass mi and trajectories of mass mj if Hand Elbow Shoulder Position ph pe ps Velocity vh ve vs • distance(motions(mi), refmi) is statically different with distance(motions(mj), refmi) Representation of possible trajectory sets 07/10/2020 Soutenance de Doctorat 33

Background Modeling dynamics Perceptual analysis Motion adaptation Biomechanical Study For each criterion: • – For each mass, One-way ANOVA using Tukey’s test (p<0. 05) – Performance = percentage of significant differences 90 Results Position • Velocity – Position-based criteria 85 80 • Not accurate enough to detect a difference 75 between different masses (~50%) Percentage of significant differences 70 – Velocity-based criteria 65 • Velociy of the hand the elbow seem to be good candidates (>75%) • Velocity of the shoulder is less relevant (~50%) 60 55 50 07/10/2020 Hand Soutenance de Doctorat Elbow Shoulder 34

Perceptual study Background Modeling dynamics Perceptual analysis Motion adaptation 13 naive subjects watched pairs of lifting motions • 3 conditions • 1. 2. 3. 07/10/2020 Real condition: comparisons of pairs of videos of real motions Virtual condition: comparisons of pairs of computer animation sequences Real x Virtual condition: comparisons of computer animation sequences with videos of real motions Soutenance de Doctorat 35

Perceptual study Background Modeling dynamics Perceptual analysis Motion adaptation Procedure • – Comparisons of a 6 kg reference motion with masses ranging from 2 kg to 10 kg Protocol • 1. 2. 3. 4. Users watch two sequences (6 kg reference and the compared mass) Users choose the motion they feel lifting the heaviest mass Users give a certainty index (Likert scale) Next pair 07/10/2020 Soutenance de Doctorat 36

Background Modeling dynamics Perceptual analysis Motion adaptation Results percentage of comparisons where the mass is considered heavier than 6 Kgs Masses in kilograms 07/10/2020 Soutenance de Doctorat 37

Background Modeling dynamics Perceptual analysis Motion adaptation Results Real and Virtual conditions • – Discrimination performances compared to 6 kg • High for 2 kg and 3 kg • Good for 4 kg and 10 kg • Difficult to evaluate for 5 kg, 7 kg, 8 kg and 9 kg Users do not accurately perceive small differences in lifted masses • Perception of lifting motions seams very close in real and virtual environments • percentage of comparisons where the mass is considered heavier than 6 Kgs Masses in kilograms 07/10/2020 Soutenance de Doctorat 38

Background Modeling dynamics Perceptual analysis Motion adaptation Results Questionnaire results • – Used velocities over positions for discrimination • Velocities at the beginning and the end of the lifting • Largely reported to use the velocity of the shoulder ≠ biomechanical parameter 07/10/2020 Soutenance de Doctorat 39

Partial conclusion Background Modeling dynamics Perceptual analysis Motion adaptation Biomechanical study: good kinematic parameters performance • – Hand elbow velocities – Other parameters should be investigated (actor skills, lifting strategies, etc) – Specific to motions lifting masses from 2 kg to 10 kg Perceptual study • – – Subjects are not able to perceive a one-kilogram difference Very close levels of accuracy for real and virtual conditions Effect of the character’s representation? Differences between subject’s comments and biomechanical study results 07/10/2020 Soutenance de Doctorat 40

Background Modeling dynamics Perceptual analysis Motion adaptation Outline Understanding relevant biomechanical parameters Modeling dynamics in human motion: measure of balance User perception of dynamic parameters 07/10/2020 Natural adaptation to dynamic constraints Soutenance de Doctorat 41

Dynamic adaptation of human motions Background Modeling dynamics Perceptual analysis Motion adaptation Goal • – Adaptations to new continuous dynamic constraints • Constraints not present in the original motion – Real-time interactions – Naturalness of the motion • Believability (users) / Correctness (dynamic equations) Previous works • – Motion synthesis: (-) design, computation time – Motion adaptation: (+) human style 07/10/2020 Soutenance de Doctorat 42

Dynamic adaptation of human motions Background Modeling dynamics Perceptual analysis Motion adaptation Previous works • – Short perturbations [Zordan 05, Komura 05] – Dynamic correctness of whole motion [Tak 00, Yamane 00, Shin 03] – Reactions to forces (carrying objects, pushes, wind, etc) [Ko 96, Aydin 99, Shin 03] Problem: how to adapt existing motions to new dynamic constraints? • – Believability vs. physically correct reactions • Sensibility to velocity vs. accelerations [Gibson 79, Werkhoven 92] Static equations • Static equations error, but low accuracy of perception skills • Assumption: acceptable error controlling static balance (using biomechanical knowledge) • Refinement introduce lost velocity information through motion retiming – In real-time: low accuracy + biomechanical knowledge 07/10/2020 Soutenance de Doctorat 43

Background Modeling dynamics Perceptual analysis Motion adaptation Overview Computation of correct interaction pose Static Footprint adaptation or generation preparation phase Interaction detected in near future 1 - Static correction of a pose 07/10/2020 MKM [Kulpa 05] Dynamic correction and motion retiming Static interaction phase motion time instant of interaction’s start 2 - Preparation to the interaction phase Soutenance de Doctorat 3 - Adaptation of the interaction 44

Background Modeling dynamics Perceptual analysis Motion adaptation Static correction of a pose Strategies for dynamic balance [Hof 07, Azevedo 06] • 1. Move the Co. P in the Bo. S 2. Rotate segments around the COM 3. Enlarge Bo. S (step, grasp object, etc) 1 - Static correction of a pose [Hof 07] [Azevedo 06] 07/10/2020 Soutenance de Doctorat 45

Static correction of a pose Assuming no accelerations: static equations • Background Modeling dynamics Perceptual analysis Motion adaptation 1 - Static correction of a pose 90 N – Error – Ankle strategy [Hof 07]: search for the correct ankle rotation Өankle that minimizes C • Linear approximation until convergence 180 N 235 N 07/10/2020 Soutenance de Doctorat 46

Preparatory phase Step 1: Computation of a correct interaction pose • Background Modeling dynamics Perceptual analysis Motion adaptation 2 - Preparation to the interaction phase 1. Compute correct static pose ensuring balance with the given forces using previous method 2. Use empirical biomechanical knowledge observed on subjects to adapt the Bo. S (using inverse kinematics [Kulpa 05]) Pushing Pulling an object 07/10/2020 Soutenance de Doctorat 47

Preparatory phase Step 2: Footprint adaptation or generation • 1. Several steps exist in the original motion step adaptation 2. Otherwise step generation Background Modeling dynamics Perceptual analysis Motion adaptation 2 - Preparation to the interaction phase Footprint adaptation • – Modify the 3 last steps of the original preparatory sequence prior to the interaction (based on [Laurent 88]) – Use MKM [Kulpa 05] to solve the new footprints Footprint generation • – Finite state machine to generate the steps When close to the interaction pose • – filtering toward the statically corrected interaction pose 07/10/2020 Soutenance de Doctorat 48

Adaptation of the interaction Interaction starts necessary to adapt the motion • 1. Static correction using previous method 2. Retiming of the action based on user specifications Background Modeling dynamics Perceptual analysis Motion adaptation 3 - Adaptation of the interaction Retiming process • – Previous experiments showed a shift of the peak of velocity when the mass ↗ • Discontinuous strategies – Users tune the style of the character • Action duration: same task can be performed in different durations • Maximum time delay ΔFmax: users set ΔFmax for the maximum force Fmax that a character can handle – To be improved by further experiments 07/10/2020 Soutenance de Doctorat 49

Results – weight lifting Captured 2 kg 07/10/2020 Captured 10 kg Soutenance de Doctorat Background Modeling dynamics Perceptual analysis Motion adaptation Captured 2 kg + 8 kg corrected 50

Background Modeling dynamics Perceptual analysis Motion adaptation Results Push Captured Push +100 N Corrected Push +200 N Quick 07/10/2020 Soutenance de Doctorat 51

Background Modeling dynamics Perceptual analysis Motion adaptation Results Various applications • – Carrying objects, pushing, pulling, turning, reacting to continuous forces, etc Real-time: whole correction < 1 ms • 10 kg – Up to 15 characters at 60 Hz 25 kg 20 kg Fc Pushing objects 07/10/2020 Compensating centrifugal force Fc Soutenance de Doctorat 52

Partial conclusion Background Modeling dynamics Perceptual analysis Motion adaptation Real-time adaptation to new dynamic constraints • – Wide range of applications: pushing, pulling, carrying objects, handling turns, etc – Reaction to continuous perturbations – Low computation time (15 characters at 60 Hz) • Most of current techniques handle only one character in real-time (30 Hz or 60 Hz) Improvements • – User perception of corrected motions – Need to validate the empirical choices with biomechanical studies Applications • – Worker carrying objects in a digital factory – Characters responding in real-time to user-defined constraints (video games) 07/10/2020 Soutenance de Doctorat 53

Conclusion Contributions • 1. None of the tested criteria succeeded in accurately detecting balance • Xco. M seems to be a promising criterion 2. Human perception of dynamic parameters seems similar in real and virtual environments, even if not accurate • Less accurate than selected kinematic criteria • Just Noticeable Difference vs. mass estimation in [Runeson 81] 3. New method for real-time dynamic adaptation of human motions • Based on static equations and a motion retiming process • Most of dynamic methods in the literature can handle at most 2 characters in realtime 07/10/2020 Soutenance de Doctorat 54

Conclusion Future work: raises challenging questions about dynamics in human motion • – Dynamic balance • How to define balance in dynamic situations • Definition of an accurate criterion (Xco. M-based? ) • Multidisciplinary studies and protocols – User perception • Studies on wider masses, various kind of motions • Effect of the actor’s skills, of the character’s representation – Dynamic adaptation • User perception of resulting motions • Biomechanical studies to validate the empirical choices (behaviors, velocity profiles, etc) 07/10/2020 Soutenance de Doctorat 55

Thanks for your attention Selected publications • L. Hoyet, F. Multon, T. Komura and A. Lecuyer (2010): Can We Distinguish Biological Motions of Virtual Humans? Perceptual Study With Captured Motions of Weight Lifting. In the Proceedings of the 2010 ACM symposium on Virtual reality software and technology, Honk-Kong, November 22 -24, 2010. • L. Hoyet, F. Multon, T. Komura and A. Lecuyer (2010): Perception based real-time dynamic adaptation of human motions. Invited paper in MIG ’ 10: The Third International Conference on Motion in Games 2010, Utrecht, The Netherlands, November 14 -16, 2010. • L. Hoyet, F. Multon, K. Mombaur, E. Yoshida (2009): Influence du choix du modèle sur l'estimation du centre de pression à partir du ZMP. Journées Nationales de Robotique Humanoïde, Nantes, Mai 2009 • F. Multon, R. Kulpa, L. Hoyet, T. Komura (2009): Interactive animation of virtual humans from motion capture data. Computer Animation and Virtual Worlds 20(5): 1 -9. • F. Multon, R. Kulpa, L. Hoyet, T. Komura (2008): From Motion Capture to Real-Time Character Animation. Lecture Notes in Computer Sciences LNCS 5277, MIG 08, p. 72 -81. •

Velocity profile Up: average maximal velocity (left) and acceleration (right) of the hand (lifting part) Down: average maximal velocity (negative) (left) and deceleration (right) of the hand (dropping part) 07/10/2020 Soutenance de Doctorat 57

Performance State of the art • – Dynamic controllers: <2 characters in real-time in [Coros 10] – Motion databases: search time 1 ms (10 motions) to 7. 5 ms (70 motions) ~2 characters at 30 Hz [Komura 05] – Dynamic filters: 28 -DOF skeleton model, ~80 ms/frame [Yamane 03, Tak 05] Our approach • – < 1 ms per frame – Up to 15 characters at 60 Hz Motion Push Pull Addit. Interaction Foot seq. Dynamic # of Force pose corr. (ms) iterations 20 N 0. 23 0. 005 0. 27 1 100 N 0. 24 0. 005 0. 35 2 200 N 0. 26 0. 005 0. 51 2. 15 20 N 0. 24 0. 005 0. 43 1. 92 100 N 0. 24 0. 005 0. 40 2 200 N 0. 24 0. 005 0. 42 1. 99 Results from a standard laptop (Dualcore 2. 33 GHz CPU, 2 GB RAM and NVidia Quadro FX 3500 M GC) 07/10/2020 Soutenance de Doctorat 58