Matt Mitchell David Pennenga John Formica Squad Tactics

Matt Mitchell David Pennenga John Formica Squad Tactics and Artificial Intelligence

Correct Movement �Topics to cover Levels of threat Straying from the team Use of caution Team context Speed of movement

Levels of Threat �The more targets, the greater the possibility of obstructing line of sight (LOS) How to fix this? �If the player is looking in the direction of NPC Move to the angle of LOS that has less targets

Levels of Threat (cont. ) �If the player is looking away from NPC should move in opposite direction that the player is looking

Levels of Threat (Cont. ) �If above heuristics fail, prioritize obstruction Threat level of enemy �A player is more likely to target: enemies with greater threat levels ▪ Obstruct view of enemies with lesser threat levels Groups of enemies over individuals ▪ Opt for obstructing view of smaller groups or individuals ▪ Threat level also should be considered

Straying from the team �NPCs should stay close to the team Set maximum radius from team �Why? Straying from the team could: ▪ Alert enemy combatants ▪ Distract player from game ▪ Trying to find lost teammates ▪ Cause failure to assist player

Use of Caution �Teammates should avoid risking the welfare of the team �NPCs should not be the first to engage an enemy Explore a room Move over a hill or around a corner This could undermine the player’s play style

Team Context �Take different play styles of the player into consideration Some games allow player to set play styles ▪ Aggressive vs passive ▪ E. G. Republic Commando ▪ “search and destroy” ▪ “form up” ▪ “hold position” �Move at the same pace as player But not obstructing LOS Consider tactically sound behavior ▪ We will cover more about this shortly

Correct Behavior �Types of behavior Use of cover Selective firing reloading

Use of Cover �Path-finding algorithm Prefer cover and the shadows ▪ Causes player to believe in tactical prowess of NPCs ▪ Adds to realism of game play ▪ Prevents accidentally alerting NPCs Take player cues If NPC behavior does not match player behavior ▪ E. G. Speed of player movement could cancel priority for cover in path-finding algorithm if player appears to be in a rush

Selective Firing �Only fire when within range and in view Controlled bursts ▪ Maintain accuracy ▪ Conserve ammunition Reaction times ▪ Should be manipulated if individual combat-related AI is too powerful ▪ Balance high accuracy with low reaction times ▪ Certain manipulations also make contextual sense: Rainbow Six style squad games ▪ Team reaction time should not be lower than enemy team’s reaction time

Reloading �Cycling aggressive NPCs Reloading NPCs fall back or take cover ▪ Return when finished �Stagger fire of NPCs So NPCs do not run out of ammunition at the same time “taking turns” at attacking ensures that player will most likely always have supporting fire

Supporting the Player �Close support of the player Protecting the rear Reporting to the player Selecting a target Responding to orders

Reporting to the Player �Protect player’s sides Report threats via audio or visual cues ▪ Messages are a staple of squad game play ▪ Do not overwhelm player ▪ Reporting threats while in combat is not usually necessary Pick appropriate engagement times ▪ Not in a fire fight ▪ Do not aggravate more enemies than already fighting

Selecting a target �Chose targets other than the player’s Chose targets farthest from the player’s LOS NPCs are there to complement game play not hinder it �Also should take threat level into account Player might need assistance ▪ Health of player’s target versus player’s own health Consider higher concentration of targets

The player is most important �Weapon selection NPC weapon should not be more powerful than player’s unless dictated by player �Weapon and item selection Allow player first choice ▪ Implemented by tracking player proximity to items ▪ If the player approaches items and walks away then NPCs may pick up items �Reaction time Allow player to attack enemies first ▪ Unless team is under attack

Implementation �Line of sight avoidance hierarchy Each child avoids the line of sight of the parent Allows for easy restructuring in the event of casualties ▪ Node replacement algorithms �Threat awareness Use LOS algorithm to find and report targets ▪ Maintain group awareness of threats by “looking around” so that NPC LOS updates, player awareness of environment ▪ Used in some groups’ Robocode this semester �Environment awareness Identify cover/restricted areas/straying from player

Implementation (cont. ) � NPCs reference a team knowledge base that governs NPC behavior � Finding available NPCs Availability = (1+N)(1+O)(1+P)+(Q*[infinity]) ▪ ▪ N= # enemies in covering area (integer value) O= # enemies within range (integer value) P= # enemies threatening team (integer value) Q= supporting another teammate (boolean value) The above formula ▪ Provides a return value of greatest value in the event that the NPC in query is supporting a teammate (optimal behavior) ▪ If the NPC is not supporting teammate, it provides a finite return value which can easily be compared with the availability of other NPCs

Example of Correct NPC Behavior �Rainbow Six: Raven Shield Team of AI finish a mission �http: //www. youtube. com/watch? v=-9 f 3 c. XBB-Nc �Visible in this video Maintaining constant awareness of environment Reaction time alterations for context ▪ Very fast for mimicking elite soldier behavior “Stick to the shadows” path-finding Controlled fire NPCs stay close to the team Appears to use LOS dominance hierarchy

Team AI and Emergent Maneuvers �Decentralized approach Squad members exchange info equally ▪ Messages! No commanding force ▪ Instead, equally shared knowledge bases for reference of each member Squad behavior is an extension of individual AI �Centralized approach Leader receives and interprets info from soldiers Orders soldier behavior ▪ Differing strength of orders ▪ This will be covered shortly

The Decentralized Approach � Pros Extension of individual AI (easier to implement) ▪ All AI simply must communicate with each other either directly (messages) or indirectly (communicating with an abstract knowledge base) Robustly handles situations ▪ Emergent behavior facilitates problem solving without having to specifically code for given tasks Deals well with variations in capabilities of teammates ▪ Individual AI is permitted to interpret or act upon supplied info differently ▪ hence this approach being an extension of individual AI Easily combined with scripted AI � ▪ Same reason as above Cons Weak at autonomy or tight coordination ▪ Lack of specific plans for problems hinders team cohesion Cannot deal with individual strengths/weaknesses

Squad Maneuvers �Tightly coordinated Members rely on detailed, repeatedly rehearsed drills, and continuous flow of info ▪ In CSE terms, events are anticipated and coded specifically to be solved �Loosely coordinated Does not rely on planned maneuvers Relies more on verbal communications

Emergent Behavior � Fire and maneuver behavior individual behavior of either firing or moving � ▪ Communicate this within the squad ▪ Emergent behavior of position cycling Goal behavior achieved via communication Stay close to cover ▪ Communicate info of cover whereabouts Prevent blocking LOS of others ▪ Communicate direction of LOS Take weapon capabilities into account ▪ Damage, area of effect, etc. Maintain group cohesion ▪ NPC location and intentions Spread out ▪ Communicate distance from other NPCs Keep LOS on targets ▪ Target locations, if the target is moving in the direction of another NPC

Squad Member’s Mental Picture � For each member Current position and activity Claimed destination position LOS � For each opponent Last known position/state Estimated current position Members engaging opponent Members able to observe opponent LOS � For other hazards/threats Known/estimated position Damage radius

Messages �Why so preferred? Model communication latency by queuing messages Present the message in game play Can filter messages for player benefit ▪ Prioritize info of messages Sends to dead members assumed alive for realism ▪ Only serves a purpose when player witnesses it Can use scripted entities to direct members ▪ The content of messages serves a purpose whether the source of the info is adaptable AI or not Accommodates human members

Ambush Example �Goals Wait for enemy to enter kill zone Pull back to predefined rally point after engagement Fall back when discovered �How does a decentralized approach handle ambushing?

Ambush Example (cont. ) �Decentralized approach… Returns fire well Can prefer being near rally point BUT ▪ Cannot reach unanimous behavior on attacking because each agent will carry out its own behavior ▪ Cannot orchestrate movement well because there is no central executive force to create a consensus of behavior

Planned Maneuvers �Centralized approach styles Authoritarian command style ▪ Focuses team performance by forcing agents to perform commands ▪ Cannot be ignored by individual AI Coaching command style ▪ Advises, rather than forces, agents ▪ Allows for agents to override commands ▪ Problems could arise from agents wanting to always perform their own behavior

The Squad’s Situation �Things to consider Force ratio ▪ ratio of allies to enemies Line of fire ratio ▪ ratio of # of enemies in allies’ line of fire to the opposite �Consider Boolean states to make judgment calls Knowledge bases First order logic conjunctions

A Good Maneuver �Chosen path is a good combination of Short travel time Concealment from pursuing enemies Sufficient space to maneuver

Maneuvering �Progress should be monitored Arriving at position informs the squad Provides individuals with a new command �Pros of this approach Allows for changes in squad behavior mid- maneuver ▪ E. G. a member is trapped in a spot which forces a reevaluation of the path finding algorithm and sends it far away from the squad

Pulling Back �Use maps or an alteration of the A* function �New cost function considers Nodes that can be fired at from preceding nodes Nodes that provide insufficient space to bypass Nodes that are a bad position from which to fight �The time and spatial complexity of this new function force it to be applied to the squad aggregate for practicality

Pulling Back (cont. ) �To properly pull back Mark each position just before a bend in the path that blocks the LOS of many preceding positions on the path Features sufficient room for members to bypass it ▪ If first choice of bypass is obstructed some how, a second choice should be available Is a good position from which to fire ▪ Pulling back across an open field versus a rocky hillside

Command Hierarchies Chain of command: Commander -> captain -> sergeant -> soldier More subdivisions are allowed if necessary � Creates an organized and efficient way of commanding soldiers �

Example: Outflanking Maneuvers �Strategic Decision Process Check if maneuver is possible/appropriate ▪ Doors are accessible (referencing door IDs so there are no duplicate actions) Compare ▪ Distance between opponent and door with distance between soldier and door �Maneuver organization Consider distance to closest door so maneuver is as fast as possible

Example: Covering Exits Maneuver �Strategic decision process Try to determine whether the maneuver is possible/advantageous ▪ Check if ally must cross a specific room in order to get from their room to a target

Example (cont. ) �Maneuver organization Find how many soldiers are needed to cover exits Order soldiers to cover designated positions Decide on a room to search Order soldiers to move to reachable doors of the first room When all of the above have been completed, a search command is sent to squad

Formations �Types of formations Line Left flank Right flank Column Box Wedge Vee

Facing in Formations �Different facing applies to different tactics Defense vs offense LOS Friendly fire Project more firepower �Staggered line �Outward facing vs inward facing

Ordering �Mixed unit type ordering Weaker/longer-ranged units in the back of the line or center of a box Faster units placed at flanks �Closest position Calculating the best match of unites to positions ▪ Sort units based on their minimum distance to the closest position ▪ Iterate through the sorted list ▪ Assign each unit the closest unused position

Unit mobility �Direction may play a role if turning is difficult �Create formation positions that avoid crossing paths of units Use movement vector and parallel paths �Arriving time Fix variations in arrival time by making all units’ paths lead to the center of the formation and order is determined by arrival time (illustration on next slide)

Unit Mobility (cont. )

Unit Mobility (cont. ) �Spacing distance Must be scaled so that the size difference in units is considered �Ranks Each formation has a maximum occupant limit ▪ Start new formation once reached �Playbook If certain places are reserved for specific units restrictions must be coded ▪ If no match found for a position, slot may be skipped (hole is made)

Moving Out �When do the units fall into formation? �Upon initial movement Might delay movement Might cause unit backtracking �Just before destination Might give player impression that nothing is happening Might be best choice for coordinated attack

Moving out (cont. ) �do formation at destination Faster movement/no delays Formations made as distance to destination decreases Not as effective while en route Not as organized upon arrival

Moving out (cont. ) �Group path finding and movement Calculate path for lead, others apply formation offsets to path Flocking style where group follows slightly behind lead in rough formation When mixed units move as a group, differences in locomotion can break formation integrity ▪ Solution: all units move like the slowest unit ▪ Problem -> faster units might appear to move in slow motion

Moving out (cont. ) �React to ambush Spread out to avoid area of effect or converge to chase down an enemy �After passing obstacle or attack regroup in the same fashion that the group was originally formed Reassign positions to make up for lost units if necessary

Influence vectors �Algorithms capable of tactical assessments in a dynamic environment �Modification of influence mapping Cells of influence are calculated from unit positions outward Total influence: a factor which assesses bot tactical situation in the engagement area Each “cell” superimposed on the influence map has a calculated value associated with agent and environmental qualities

Genetic Programming �Learning algorithms that give an NPC the ability to adapt to opponents’ playing styles in real time �Pros Provide more challenge as opponents because they are unpredictable Facilitate the discovery of new strategies �Cons Developers cannot test what behaviors the NPCs will exhibit

Genetic Programming (cont. ) � Entire team of NPCs view as one chromosome Each agent derived from a different part of the chromosome ▪ Creates heterogeneous teams � Five node sets, total of fifty nodes used in evolution

Genetic Programming (cont. ) �Fitness calculation Takes into consideration the game’s duration and remaining health of team and enemy agents �Selection Elitism: m copies of the best n chromosomes from each generation are copied directly into the next generation Roulette wheel selection ▪ Any chromosomes selected are subject to randomness

Example of capabilities �http: //www. youtube. com/watch? feature=pla yer_embedded&v=4 Er. EBkj_3 PY