AI to Control Pacing in Games Borut Pfeifer

AI to Control Pacing in Games

Borut Pfeifer • White Knuckle Games – Reality’s Edge, 2001 -2003 • Radical Entertainment – 2003 • Articles in Game Programming Gems 4, various others

Motivations & Goals • Bad AI Programmer, No Twinkie – Max Payne, Medal of Honor. Well paced, but too heavily scripted. • Keep strong pacing without sacrificing replayability. • Take user’s skill level into account to maintain flow, for a more consistent experience. • Eliminate the need for arbitrary static difficulty levels & the development they involve – who likes picking easy anyway? • Create strong pacing while allowing user to roam freely.

Dramatic Tension Measure of complication Climax Dénouement Tension Time

Dramatic Tension In reality… Climax Dénouement Tension Time

Dramatic Tension Brenda Laurel - Computers As Theater Drama as a model for interactivity: • Enactment - The dramatic model focuses on action, which also defines interactivity • Intensification – incidents can be selected and arranged to condense time. • Unity of action – causally linked actions with a strong central action

Flow Mihaly Csikszentmihalyi - Flow: The Psychology of Optimal Experience Flow – the state in which a person is so involved in an activity that nothing else seems to matter Studied thousands of people, including experts (chess masters, artists, athletes, etc. ) to help shed light on the flow experience. Used “experience sampling” – subjects wore pagers and had to write down how the felt and what they were thinking when they received pages. Over a hundred thousand samples have been collected.

Characteristics of Flow • Challenging activity that requires skills Stimulates some part of the brain (spatial, visual, logic, memory, attention, etc. ). • Merging of action and awareness What you’re paying attention to consists of things related to achieving your goals. • Clear goals and feedback Player always knows what they need to accomplish, and is receiving continuous feedback as to how well goals are being achieving • Concentration on the task at hand A high degree of concentration must be required to achieve the goals.

Characteristics of Flow • Loss of self-consciousness Because of complete concentration on the task at hand, you stop thinking about yourself, problems at home or work. • Transformation of time – compression or expansion. Doesn’t matter if time is a factor in the experience. • The paradox of control Typically, the person experiencing flow feels like they have high degree of control over their activity. It’s more the possibility of control, not the actuality of having it. People don’t enjoy the sense of control, but of exercising control in difficult situations.

Flow – Frustration vs. Boredom Frustration Challenge B D A C Skill A – The user starts in flow state Flow Channel Boredom

Flow – Frustration vs. Boredom Frustration Challenge B D A C Flow Channel Boredom Skill B – The user is learning game controls, is not yet skilled, the game challenges him too quickly, and he gets frustrated. He must learn new skills to return to flow.

Flow – Frustration vs. Boredom Frustration Challenge B D A C Flow Channel Boredom Skill C – The user has learned how to play the game, or starts out adept at it, low challenge leads to boredom. In order to return to flow, the challenge must be increased).

Flow – Frustration vs. Boredom Frustration Challenge B D A C Flow Channel Boredom Skill D – The user returns to flow having improved skills and being able to function at a higher level of challenge.

Application • Measure the player’s skill level to figure out roughly where in the flow zone the player might be. • Increase challenge corresponding with improvements in the player’s skill. • Also adjust challenge based on desired dramatic tension.

System Components • Dynamic game elements What about the game can change from moment to moment that affects the difficulty? • Definition of difficulty progression Where are the hard spots and easy spots? • Measurement of the player’s skill • Information to tie dynamic elements together There may be an aesthetic component to selecting groups of dynamic game elements. • Sequencer that uses all these elements Selects the elements necessary to achieve the desired difficulty with respect to the player’s skill.

System Components Pool of available game elements Difficulty Progression Logic Player actions Player Skill Measurement Element Grouping Information Difficulty Sequencer Created game elements

Dynamic Difficulty Elements For an action/combat game, these would be different types of enemies, their weapons, etc. Need to evaluate the difficulty of an individual element. Designer defined difficulty rating – allows quick heuristic evaluation to compensate for a variety of complex factors. Simpler to tune. More complex mechanic may be needed If an element is actually comprised of sub-elements (an enemy type with a weapon, other power-ups), may need a rating for each sub-element. Difficulty rating scale Needs to be defined to correspond with player skill measure, 1 (easiest) to 10 (hardest) for example.

Defining Difficulty Progression – Two Approaches Offline – Designers provide hints to the system where the easy and hard spots are. Example – Location based difficulty Designers create areas using trigger volumes and associate a difficulty with an area, as well as a list of available enemy types, spawn points, etc. Mechanic based - Model difficulty based on some element of the game’s experience Example – Amount of remaining gameplay As the player gets closer to completing all the goals in a level, the system would modulate difficulty as necessary.

Player Skill We need a measure of the player’s skill to determine the experienced difficulty. Factors to consider: • Time (valid for most action-related games). • Difficulty of elements the player has completed. • Number of elements the player is in the process of completing.

Player Skill For a combat oriented game, these would translate to the following: • Time to defeat a single enemy – indirect relationship • Designer defined difficulty value for that enemy type – direct relationship • Number of other enemies the player is currently engaging in combat – direct relationship

Player Skill Basic Player skill measure = Enemy difficulty / Time to defeat enemy Would be nice to have same scale as enemy difficulty: So we add a constant that represents a basic unit of difficulty: the time it would take a typical novice player (at skill level 1) to defeat a simple enemy (at difficulty level 1), with the simplest available tools (weapon and any other power-ups) the first time he plays the game. Player skill measure = Enemy difficulty / (Time to defeat enemy * basic unit of difficulty)

Player Skill Need to factor in other ongoing challenges the player is dealing with at the time. Scale enemy difficulty by number of other opponents Another constant, based on usability testing or game design goals. Example – Set scale per # of enemies Scale the player skill measure by a constant amount per number of enemies: +1 enemy: x 1. 125, +2 enemies: x 1. 25, +3 or more enemies: x 1. 5 - If the player has to defeat three enemies at once, defeating any 1 of them is harder (by 1 ½ times) than if the player faced that enemy alone. Final Player skill measure = (Enemy difficulty * scale factor for number of simultaneous enemies) / (Time to defeat enemy * basic unit of difficulty)

Player Skill – Other Factors to Consider • Use a simple average (not weighted by recency) of skill measures over the last few minutes of gameplay. • Prevents player from cheating (quickly changing performance to cause the system to lower difficulty). • Difficulty modulation should be invisible to the player, otherwise it would removed them from the experience. • Simpler is better - easier to tune. • Time is often the key factor in skill measurement for a game with any sort of action component. • Avoid additional factors (weapon accuracy, damage dealt) that are not directly associated with effectiveness.

Grouping Difficulty Elements One last bit of information needed, potentially, before selecting elements to define the goal difficulty. How do we select multiple difficulty elements in a pleasing fashion? How do we connote a causal relationship between elements? Usually would come in the form of additional designer input to the system, to control the aesthetics of grouping and content-specific relationships.

Difficulty Sequencer Goal difficulty = Player’s skill * difficulty modifier Difficulty modifier comes from the defined difficulty progression at that location/time (very hard, normal, easy, etc. ) Now randomly select from the difficulty element groupings Select from the available list of elements to match the desired characteristics of the grouping, whose total difficulty sums to the goal difficulty.

Difficulty Sequencer - Example So if we have 5 difficulty ranges, each range might have these modifiers: Very Hard - 2. 0 Hard - 1. 5 Normal - 1. 0 Easy - 0. 75 Very Easy - 0. 5 If the player’s skill is at 8, and we’re in a very hard section, then the goal difficulty is 16. Note: discrete difficulty settings

Difficulty Sequencer - Example Let’s say we have the following groupings of enemies: • One very strong enemy • Two enemies, one stronger than the other • Two enemies of equal difficulty • Three enemies, one stronger than the other two • Three or more weak enemies And the following enemies available in this area: • Commando - Difficulty 10 • Sniper - Difficulty 5 • Infantry - Difficulty 4 • Guard dog - Difficulty 2

Difficulty Sequencer - Example Randomly select the two enemies of equal strength group The system can’t create two enemies of equal strength within a small enough range of the goal - 20 (10+10) or 10 (5+5). So then select another grouping, two enemies, one of greater strength. It chooses an enemy of difficulty 10 and one of difficulty 5 Their total difficulty is sufficiently close to the goal difficulty. Now it creates the enemies according guidelines specified by the designers (at applicable spawn points, with available weapons/armor).

Difficulty Sequencer – More Factors The game may require more complex difficulty model – enemies may pick up enhancements that greatly affect their strength (weapons, armor, etc. ) Each individual sub-element then has it’s own difficulty. The sequencer would just have to consider each valid combination of subelements (enemy A with weapon 1, enemy A with weapon 2, etc. ). The difficulty would just be the sum of the individual difficulty elements.

Strengths • Simpler tuning process – don’t have to tweak each enemy’s position, type, etc. • Speed & ease of development – well suited for user mods (Never. Winter Nights) • Can create stronger, more consistent, pacing even in nonlinear environments • Eliminates need for static difficulty levels by taking into account player’s skill throughout the game. • Enhances replayability by avoiding heavily scripted pacing

Weaknesses • Relies on semi-arbitrary, heuristic based input to define difficulty scales. • Must keep mechanics simple, otherwise it can be hard to tune (since they only indirectly affect the player’s experience). • Difficulty mechanics must be as invisible to the player as possible.

Further work • Keep track of the rate at which player’s skill is changing – a measure of how much they’re learning. • Could add requirements for skills dynamically - If the player’s skill set isn’t growing, add tasks that require new skills to succeed. • As the dynamic elements become more complex, the selection/grouping information becomes more involved.

Feedback - bpfeifer@radical. ca References Laurel, Brenda, Computers As Theater, Addison Wesley, 1991 Csikszentmihalyi, Mihaly, Flow: The Psychology of Optimal Experience, Harper Collins, 1990 Mateas, Michael, and Stern, Andrew, “Façade: A One Act Interactive Drama, ” Game Developer’s Conference 2003