Learning Reactive Behavior in Autonomous Vehicles SAMUEL Sanaa

Learning Reactive Behavior in Autonomous Vehicles: SAMUEL • Sanaa Kamari

SAMUEL • Computer system that learns reactive behavior for autonomous vehicles. – Reactive behavior is the set of actions taken by an AV as a reaction to sensor readings. • uses Genetic algorithm to improve decision making rules. • Each individual in SAMUEL is an entire rule set or strategy.

Motivation for SAMUEL • Learning facilitates extraction of rules from the expert. • Rules are context based => impossible to account for every situation. – Given a set of conditions, the system is able to learn the rules of operation from observing and recording his own actions. • Samuel uses a simulation environment to learn.

SAMUEL • Problem specific module. – The world model and its interface. – Set of internal and external sensors – Controllers that control the AV simulator – Critic component that criticizes the success or failure of the AV. [1]

SAMUEL (cont) Performance module – Matches the rules. – Performs conflict resolution. – Assign some strength values to the rules. • Learning module. – Uses GA to develop reactive behavior, as a set of condition -reaction rules. • GA searches for the behavior to exhibit the best performance – Behaviors are evaluated in real world model. – Behaviors are selected for duplication and modification. [1]

Experiment Domain: Autonomous Underwater Vehicle navigation and collision avoidance • Training the AUV simulator by virtually positioning it in the center of a field with 25 mines, and an objective outside the field. • 2 D AUV must navigate through a dense mine field toward a stationary object. • AUV Actions: set speed and direction each decision cycle. • System does not learn path, but a set of rules that reactively decide a move at each step.

Experiment Results • Great improvement in both static and moving mines. • SAMUEL shows that reactive behavior can be learned. [1]

Domain: ROBOT Continuous and embedded learning • To create Autonomous systems that continue to learn throughout their lives. • To adapt a robot’s behavior in response to changes in its operating environment and capabilities. • experiment: robot learns to adapt to failure in its sonar sensors.

Continuous and Embedded learning Model • Execution module: controls the robot’s interaction with its environment. • Learning module: continuously tests new strategies for the robot against a simulation model of its environment. [2]

Execution Model • Includes a rule-based system that operates on reactive (stimulus-response) rules. – IF range = [35, 45] AND front sonar < 20 AND right sonar > 50 THEN SET turn = -24 (Strength 0. 8) • Monitor: Identifies symptoms of sonar failure. – measures output of sonar, compare it to recent readings and direction of motion. – Modifies simulation used by learning sys to replicate failure.

Learning Module • Uses SAMUEL: uses Genetic algorithm to improve decision making rules.

Experiment • Task requires Robot to go from one side of a room to the other through an opening. • Robot placed randomly 4 ft from back wall. • Location of opening is random. • Center of front wall is 12. 5 ft from back wall

Experiment (cont) • Robot begins with a set of default rules for moving toward the goal. • Learning starts with simulation that includes and all sonars working. • After an initial period one ore more sonars are blinded. • Monitor detects failed sonars, learning simulation is adjusted to reflect failure. • Population of competing strategies is re-initialized and learning continues. • The online Robot uses the best rules discovered by the learning system since the last change to the learning simulation model,

Experiment Results • Robot in motion with all sensors intact: – a) during run and b) at goal. • Robot in motion after adapting to loss of three sensors: front, front right and right: – a) during run, and b) at goal. [2]

Experiment Results [2] • a) Robot with full sensors passing directly through doorway. • b) Robot with front sonar covered. • c) Robot after adapting to covered sonar. It uses side sonar to find opening, and then turns into the opening.

References • [1]. A. C. Schultz and J. J. Grefenstetts, “Using a genetic algorithm to learn reactive behavior for autonomous vehicles, ” in Proceedings of the AIAA Guidance, Navigation, and Control Conference, (Hilton Head, SC), 1992. • [2]. A. C. Schultz and J. J. Grefenstetts, ”Continuous and Embedded Learning in Autonomous Vehicles: Adapting to Sensor Failures”, in Proceeding of SPIE vol. 4024, pg 55 -62, 2000.