Finite State Machines GAM 376 Robin Burke Winter

Finite State Machines GAM 376 Robin Burke Winter 2006

Outline ¢ Finite State Machines Theory l Implementations l Homework #2 ¢ Break ¢ Lab ¢

AI in the Game Loop AI is updated as part of the game loop, after user input, and before rendering ¢ There are issues here: ¢ Which AI goes first? l Does the AI run on every frame? l Is the AI synchronized? l

AI Update Step ¢ ¢ ¢ The sensing phase determines the state of the world l May be very simple - state changes all come by message l Or complex - figure out what is visible, where your team is, etc Game The thinking phase decides Engine what to do given the world l The core of AI The acting phase tells the animation what to do l Generally not interesting AI Module Sensing Thinking Acting

AI by Polling The AI gets called at a fixed rate ¢ Senses: It looks to see what has changed in the world. For instance: ¢ Queries what it can see l Checks to see if its animation has finished running l And then acts on it ¢ Why is this generally inefficient? ¢

Event Driven AI ¢ ¢ ¢ Event driven AI does everything in response to events in the world l Events sent by message (basically, a function gets called when a message arrives, just like a user interface) Example messages: l A certain amount of time has passed, so update yourself l You have heard a sound l Someone has entered your field of view Note that messages can completely replace sensing, but typically do not. Why not? l Real systems are a mix - something changes, so you do some sensing

Finite State Machines (FSMs) ¢ ¢ ¢ A set of states that the agent can be in Connected by transitions that are triggered by a change in the world Normally represented as a directed graph, with the edges labeled with the transition event Ubiquitous in computer game AI You might have seen them in theory of computation (or compilers)

Classic Application: Regular Expressions ¢ Any regular expression can be translated into a finite state machine l ¢ this is part of what it means to be a regular expression Example (Perl) l De. Paul course ids • [A. . Z]{2, 3}([1. . 6]dd) • GAM 376

What would the machine look like?

Quake Bot Example ¢ ¢ ¢ Types of behavior to capture: l Wander randomly if don’t see or hear an enemy l When see enemy, attack l When hear an enemy, chase enemy l When die, respawn l When health is low and see an enemy, retreat Extensions: l When see power-ups during wandering, collect them Borrowed from John Laird and Mike van Lent’s GDC tutorial

Example FSM ~E Attack E, ~D ~E D Spawn D l E States: l D E Wander ~E, ~S, ~D ¢ ~S D S l Chase S, ~E, ~D ¢ Events: l l l ¢ E: enemy in sight S: sound audible D: dead E: see an enemy S: hear a sound D: die Action performed: l l On each transition On each update in some states (e. g. attack)

Example FSM Problem ~E ~E D E Spawn D ~S l l S E States: l D E Wander ~E, ~S, ~D ¢ Attack E, ~D D S Chase S, ~E, ~D ¢ E: enemy in sight S: sound audible D: dead Events: l l l E: see an enemy S: hear a sound D: die Problem: Can’t go directly from attack to chase. Why not?

Better Example FSM ~E Attack E, ~S, ~D D S E ~E D S D E Wander ~E, ~S, ~D ~S ~S D Attack-S E, S, ~D ~E ¢ l l E Chase S, ~E, ~D States: l ¢ Events: l l Spawn D S l ¢ E: enemy in sight S: sound audible D: dead E: see an enemy S: hear a sound D: die Extra state to recall whether or not heard a sound while attacking

Example FSM with Retreat Attack-E E, -D, -S, -L Attack-ES E, -D, S, -L S -S L Retreat-S -E, -D, S, L L -L • States: E -E Wander-L -E, -D, -S, L L E E -L -L E L Retreat-ES E, -D, S, L -S -L S -E Wander -E E -E, -D, -S, -L D D Spawn D (-E, -S, -L) S Chase -E, -D, S, -L Retreat-E E, -D, -S, L – – E: enemy in sight S: sound audible D: dead L: Low health • Worst case: Each extra state variable can add 2 n extra states • n = number of existing states

Augmented FSM ¢ Typically, there will be book-keeping to do when transitioning between states l For example • "direction of sound" variable • cleared when sound is absent and changing to "Wander" state ¢ Most FSM implementations allow specification of l update • what to do during each time increment in this state l enter • what to do to start up this activity l exit • what to do to end this activity

Example ¢ Chase l Enter • • l Play animation "weapon forward" Play sound "battle cry" Set heading "direction of sound" Set speed "run" Update • Set heading "direction of sound" • Move l Exit • Play animation "hand to ear" • Play sound "Huh? " • Set speed "walk"

Exercise ¢ FSM for Blinky

Hierarchical FSMs ¢ ¢ What if there is no simple action for a state? Expand a state into its own FSM, which explains what to do if in that state Some events move you around the same level in the hierarchy, some move you up a level When entering a state, have to choose a state for it’s child in the hierarchy l l l Set a default, and always go to that Or, random choice Depends on the nature of the behavior

Hierarchical FSM Example Wander Attack ~E E ~S Pick-up Powerup Chase S D Start Turn Right Spawn ~E ¢ Go-through Door Note: This is not a complete FSM l l All links between top level states still exist Need more states for wander

Non-Deterministic FSM (Markov Model) ¢ Attack ¢ Approach Aim & Slide Right & Shoot . 3 Aim & Slide Left & Shoot ¢ . 3. 4 . 3. 3 Start . 4 Aim & Jump & Shoot ¢ ¢ Adds variety to actions Have multiple transitions for the same event Label each with a probability that it will be taken Randomly choose a transition at run-time Markov Model: New state only depends on the previous state

Push-down State Machines ¢ ¢ Suppose we have some repeated behavior l common to many states l don't want to "forget" what we were doing Example l change weapon if out of ammo • might want to do this while wandering, chasing, or attacking l if we make this a state • where do we transition after doing it? ¢ Separate global state l Transition into this state temporarily • and pop back to origin

Efficient Implementation event ¢ ¢ Compile into an array of state-name, event state-namei+1 : = array[state-namei, event] Switch on state-name to call execution logic Hierarchical l l Create array for every FSM Have stack of states • Classify events according to stack • Update state which is sensitive to current event ¢ Markov: Have array of possible transitions for every (statename, event) pair, and choose one at random state

OO Implementation States as objects ¢ Actions as methods ¢ State machine can be very generic ¢ More in a minute ¢

FSM Advantages ¢ ¢ ¢ Very fast – one array access l think GBA Expressive enough for simple behaviors or characters that are intended to be “dumb” Can be compiled into compact data structure l Dynamic memory: current state l Static memory: state diagram – array implementation Can create tools so non-programmer can build behavior Non-deterministic FSM can make behavior unpredictable

FSM Disadvantages ¢ Number of states can grow very fast l Exponentially with number of events: s=2 e Number of arcs can grow even faster: a=s 2 ¢ Limited representational power ¢ Propositional representation ¢

Varieties of representation ¢ ¢ Propositional logic: l Statements about specific objects in the world – no variables l Jim is in room 7, Jim has the rocket launcher, the rocket launcher does splash damage l Go to room 8 if you are in room 7 through door 14 Predicate Logic: l Allows general statement – using variables l All rooms have doors l All splash damage weapons can be used around corners l All rocket launchers do splash damage l Go to a room connected to the current room

Representation in FSMs ¢ Can only handle propositions l ¢ Difficult to add comparative tests l l ¢ each transition predicated on a proposition being true or false “pick up the better powerup” “attack the closest enemy” Expensive to count l Wait until the third time I see enemy, then attack • Need extra events: First time seen, second time seen, and extra states to take care of counting

Implementations switch statement l we can implement a FSM with a switch statement ¢ Each state is a branch switch case CHASE: ¢ if ENEMY state = ATTACK else if ~SOUND state = WANDER else Do. Chase() case WANDER: etc. ¢ Concise l but a maintenance nightmare

Transition table ¢ For each state and event l record next State Event New. State Update. Fn Chase Enemy Attack Do. Chase() Chase ~Sound Wander Do. Chase()

Buckland ¢ Uses the "State" design pattern l l ¢ all information relative to an object's state is encapsulated changing state is a matter of changing this object Calling sequence l l l Game -> Object "update yourself" Object -> State "update me" many advantages • state objects can be shared without duplicating code • easy to store "previous state" information

Overall design Game-wide state machine ¢ When each object is updated, ¢ the state is loaded into the machine and processed l new state is recorded with object l ¢ Assumes that everything in the game uses a state machine

Structure view

Singleton pattern ¢ ¢ All states are unique instances l only a single copy of the "Quench Thirst" state Benefit l no dynamic allocation and destruction of state objects Drawback l no object-specific information can be stored in the state Technique l use static allocation l use a static member function that always returns this instance l make constructor private

Example class Quench. Thirst : public State<Miner> { private: Quench. Thirst(){} Quench. Thirst(const Quench. Thirst&); Quench. Thirst& operator=(const Quench. Thirst&); public: static Quench. Thirst* Instance(); virtual void Enter(Miner* miner); virtual void Execute(Miner* miner); virtual void Exit(Miner* miner); }; Quench. Thirst* Quench. Thirst: : Instance() { static Quench. Thirst instance; return &instance; }

Improved design ¢ Each object may have slightly different requirements for how its state should be handled l some may not need a state machine l some may have something more complex l • hierarchy, Markov, etc. ¢ Give each object its own state machine

Next version

Note ¢ That this state machine has a global state and previous state very limited memory l to handle state "blips" l

Messaging ¢ ¢ Perception by polling is inefficient l never want to be "busy waiting" Example l if guard does nothing until player enters l guard should not be constantly checking "did player enter" • in order to transition to next state ¢ Better method l publish / subscribe l have player send a message to each item in a room when he enters l state transition can be predicated on this message

Note ¢ Messages sent to specific recipients not hard to imagine an extension for localized messages l global messages very expensive l Modern games use messaging extensively ¢ Need for unique entity ids and entity registry ¢

Homework #2 ¢ We are not using West. World l ¢ too weird Instead AIsteroids l an asteroids game l you will be working with the AI controlling the ship l

Part I ¢ Add a new state when there are few asteroids l go into powering-up state l try to avoid asteroids and just get lots of power-ups l sort of a specialized idle mode l

Part II ¢ Add toroidal geometry the AI does not measure distances right l off-screen wrap-around not used l ¢ Change the code to do this right

Major hint pretend asteroid is at position (x, y-screen. H) wrap region screen

Break