CAP 6938 Neuroevolution and Developmental Encoding Overview of

CAP 6938 Neuroevolution and Developmental Encoding Overview of NEAT and CPPNs Dr. Kenneth Stanley August 28, 2006

Why an Overview? • • Preparation for declaring projects Get an idea what is possible Sufficient background without details Technical details coming later in semester

NEAT: Neuro. Evoluton of Augmenting Topologies • Evolutionary algorithm applied to evolving artificial neural networks (ANNs) • Evolution (i. e. genetic algorithms) is a way to optimize ANN topology and weights • NEAT evolves increasingly complex ANNs • Able to solve difficult control and decisionmaking tasks

Evolution: Survival of the Roundest Gen 1 Select as parents Gen 2 Select as parents Gen 3 Champ!

Neuro. Evolution • Evolve ANNs • Evaluated for performance on a task – – – – Controlling a vehicle or robot Drawing a picture Recognizing an image Warning about danger Playing a board game Playing a video game Playing music • The fittest survive and reproduce

ANNs can Sense and React

NEAT Features • NEAT is unique: Networks complexify over evolution Crash Warning Champ • So they can find the right complexity for the task • NEAT solves problems with systems of diverse networks using historical marking and speciation • A first step in evolving complex systems

NEAT Applications • NEAT makes new applications possible and has set performance records Double Pole Balancing Record Vehicle Warnings Keepaway Record NERO: Real-time Neuroevolution in a Video Game Robot Duel Go Hopper

Not Just About Control and Classification Interactive Picture Evolution Interactive Drum Pattern Evolution Interactive Particle Effect Evolution Interactive Chord Progression Evolution Guitar Effect-Pedal Emulation

Non-neural NEAT • ANNs aren’t the only thing that can complexify • Anything with arbitrary structure can potentially be evolved – Cellular Automata – Lego Structures – FSMs – L-systems L-System Plant Geometry Generator by Hung-Wen Chen

What Can’t NEAT Do? • “Regular” NEAT is a direct encoding • It doesn’t encode regularities or symmetries • Regularity and symmetry are believed to be essential to representing and searching complex structures: gene reuse • Giant complex structures in nature are composed of patterns • Patterns have regularities • Indirect encodings capture such information efficiently

Indirect Encoding • Most are developmental encodings – Based on embryo to adult development – “Artificial Embryogeny” – This is how encoding works in nature • Many have been proposed • None close to natural complexity

Proposed Indirect Encodings

What’s missing? • None implement complexification as in NEAT: maybe genes can be added and marked like in NEAT – For example: L-system NEAT? • Something’s still missing…

A Recent Discovery • Drawn by a NEAT-evolved neural network!?

Compositional Pattern Producing Networks (CPPNs) • Networks of functions composed to produce patterns with regularities • An abstraction of natural development without developmental simulation • Complexifying repeating patterns, regularities, symmetries, repetition with variation, etc.

CPPN-NEAT • • A new kind of indirect encoding Elegantly captures regularities Convert patterns into phenotypes An entirely new research area Exploiting Regularity Without Development by Kenneth O. Stanley, To appear in: Proceedings of the AAAI Fall Symposium on Developmental Systems. Meno Park, CA: AAAI Press, 2006 (8 pages)

Music is a Pattern over Time • A CPPN is a pattern generator that can produce repetition with variation, just like music • A CPPN can produce several related patterns from an underlying simpler pattern (multiple instruments) • Music generation is a promising demonstration

Homework for 8/30/06 • Neural network weight-learning algorithms Fausett pp. 39 -80 (in Chapter 2) Fausett pp. 289 -316 (in Chapter 6) Chapter 1 of Sutton (on RL): Online Book ch. 1 Optional: Kaelbling RL survey paper • Remember proposals/partners due 9/11