Gene Synth Sound Synthesis by Genetic Algorithm Introduction

Gene. Synth Sound Synthesis by Genetic Algorithm

Introduction • Genetic Algorithms (GA) – Search Algorithm • Uses Darwin-inspired “survival of the fittest” heuristic. – Used In a wide range of fields • TSP, Financing, Music • But not for sound synthesis – Search space is too large

Gene. Synth • Experiment in using Gas to synthesize useful sounds • Uses an input sound as a target to model as a heuristic • Motivation – Musical – Analysis

Goal 1: Sound Fitting • Take an audio file and fit other user-specified sounds using the audio file as a guide • Example: – Beethoven’s Fifth symphony, 1 st movement as a target – Coin-jingling sounds as specified sounds – Result: Coin samples placed to sound as close as possible to the symphonic version.

Goal 2: Family Tree Exploration • Many search algorithms have a history associated with each solution • GAs are no exception • A GA solution’s history resembles a family tree – Because GAs create solutions by pairing two other solutions together • The ancestors and siblings of a good solution may yield interesting sounds for composers.

Goal 3: Modelling • If a perfect or near-optimal solution is reached, it can be used as a model – The model can then be modified in an interesting way and resynthesized to create a new useful result • But how easily will a perfect model be reached? – It depends on the sounds we select for use in the algorithm, as well as the efficiency of the GA. • If we use a sinusoidal unit generator, the problem begins to resemble the phase vocoder.

The classical GA • Origins in the mid 80 s – Many variants have since been created. – Gene. Synth is one of these variants – The classical GA must be understood to appreciate the variants

ＧＡ Overview and Terms • Chromosome: a string of data which represents a solution. Example: – 00001011 (a set of 8 bits that represents the number 11) • Individual: the solution represented by a chromosome – ‘ 11’ in the above solution. • Population: a set of chromosomes. Example: – {00001011, 00100011, 10110000} (three chromosomes)

More GA Terms. . • Mutation function: a function that changes sections of the chromosome based on some probability. Example: – M(01001011) -> 00001010 • (Two bits are flipped, changing the chromosome) • Crossover function: a function that takes two chromosomes and combines them together to result in two new chromosomes – C(10001010, 00001111)-> 10001111, 00001010 • (The original chromosomes swap halves)

GA terms and overview • Fitness Function: a • Psuedocode: Create Initial Population function that assigns a Repeat score to an Individual, Score Each Individual based on how good a Select pairs of good-ranking individuals solution it is. – If the problem is to find the number close to 77, a fitness function could be: • F(x) = | 77 – x | Mutate the selected individuals Crossover the individuals Until goal score is met

A simple example • Suppose our problem is to find the number 77. – We will use the encoding AND functions in the above examples. – We randomly create a set of 8 bit strings to start off the population of size four. • {0010, 11001010, 00010000, 11110000} – We then score each individual and create the next four via mutation and then crossover. – This process Is repeated until we have found the number 77.

Classical GA vs Gene. Synth • Fixed Length Chromosome • Independent chromosome structure • Fixed Mutation and crossover probabilities • Variable Length chromosomes • Self-referential chromosome structure • Mutation and Crossover Probabilities vary over evolution

Chromosome Structure • A bottom up example • A Sound. Root is a unit generator, which takes parameters and creates audio by some method – Ex. Sin(freq, phase), Audio. Files(File. Name) • A Transformation takes existing audio and parameters and processes it – Ex. Distortion, Normalization, Amplitude Modulation • Sound. Roots and Transformations are the basis of sound generation for Gene. Synth

Chromosome Structure • A Sound. Cell is a unit that contains one Sound. Root, and any number of Sound. Cells and Transformation Sound. Cell – Note the recursive definition. – A Sound. Cell specifies a sound entity that will be used some number of times in the synthesis Transformations Sound. Root Child Sound. Cells

Chromosome Structure: Genes • A Cell. Def. Gene is a set of Sound. Cells. Cell. Def. Gene Sound. Cells • A Place. Gene is a set of several pieces of information – A reference to a Cell In the Cell. Def. Gene, – A time and volume to place the sound Two Place. Genes 1. 4 secs -6 db 3. 12 secs -8 db

The chromosome structure • So, a chromosome is a Cell. Def. Gene and some number of Place. Genes. Chromosome Cell. Def. Gene Place. Genes 1. 4 s 1. 1 s 0. 4 s 1. 4 s 3. 1 s 2. 8 s -6 db -8 db -2 db -8 db -6 db

Chromosome to Individual • If we realize every Sound. Cell at the times and levels specified by the Place. Genes, we have generated our individual, an audio file. Chromosome Cell. Def. Gene Place. Genes 1. 4 s 1. 1 s 0. 4 s 1. 4 s 3. 1 s 2. 8 s -6 db -8 db -2 db -8 db -6 db

Fitness Function • The fitness function uses a target soundfile, and compares the individual soundfile against it. This is done by – Taking FFTs of both, and comparing them • Sums the decibel rating of the ratio of each bucket in the FFT. – Taking the RMS value of both and comparing them

Mutation • A chromosome mutates by randomizing some part of it. In Gene. Synth, – a new Sound. Cell can be created or deleted, Its parameters randomized, or one Sound. Cell might attach another to it. – A Place. Gene will mutate by changing the Sound. Cell it refers to, or by changing its amplitude or time. Place. Genes Cell. Def. Gene 1. 4 s 1. 1 s 0. 4 s 1. 4 s 3. 1 s 2. 8 s -6 db -8 db -2 db -8 db -6 db Cell. Def. Gene Place. Genes 1. 4 s 1. 1 s 0. 4 s 1. 4 s 3. 1 s 2. 8 s -6 db -8 db -2 db -8 db -6 db -12 db

Crossover • Pick two genes and taking a random number of Sound. Cells and the Place. Genes that refer to them to make one new Chromosome, and leaving the remainder to make another. 1. 4 s -6 db 1. 1 s -8 db 0. 4 s -2 db 1. 4 s -8 db 3. 1 s -6 db 2. 8 s -6 db 3. 1 s -8 db 2. 1 s -9 db 2. 5 s -7 db 0. 7 s -2 db 1. 1 s -7 db 1. 6 s -4 db 1. 4 s -6 db 0. 7 s -2 db 1. 1 s -7 db 1. 6 s -4 db 3. 1 s -6 db 2. 8 s -6 db 2. 1 s -9 db 2. 5 s -7 db 0. 7 s -2 db 1. 1 s -8 db 0. 4 s -2 db

Example of evolution score: score: score: 0. 110832, Adam Joaquin, 9 cells, 22 genes 0. 220700, Jill Hayashi, 2 cells, 42 genes 0. 012515, Paul Berio, 16 cells, 3 genes 0. 150242, Nimrod Palestrina, 8 cells, 46 genes 0. 303568, Rebecca Silverstone, 15 cells, 111 genes 0. 252139, Alicia Ligeti, 9 cells, 95 genes 0. 384353, John Shakesphere, 15 cells, 93 genes 0. 245086, Kyoko Joyce, 2 cells, 107 genes 0. 073234, Rebecca Palestrina, 2 cells, 40 genes 0. 363259, Leonardo Joyce, 13 cells, 105 genes Best in generation 1 is 0. 384353 score: score: score: 0. 384353, John Shakesphere, 15 cells, 93 genes 0. 328594, Samuel Terentino, 2 cells, 125 genes 0. 295409, William Shakesphere, 15 cells, 94 genes 0. 407776, Eve Silverstone, 15 cells, 110 genes 0. 344386, Jack Terentino, 5 cells, 62 genes 0. 168303, Albert Athene, 7 cells, 74 genes 0. 274905, Akita Terentino, 5 cells, 112 genes 0. 366032, Jill Shakesphere, 12 cells, 78 genes 0. 387343, Ikroop Shakesphere, 18 cells, 108 genes 0. 332004, Rebecca Shakesphere, 17 cells, 110 genes Best in generation 2 is 0. 407776 score: score: score: 0. 407776, Eve Silverstone, 15 cells, 110 gene 0. 062087, Akira Shakesphere, 3 cells, 13 genes 0. 448460, Vincent Shakesphere, 19 cells, 208 genes 0. 291499, Jack Terentino, 5 cells, 76 genes 0. 287326, Alicia Joyce, 7 cells, 127 genes 0. 458743, Akira Palestrina, 7 cells, 86 genes 0. 395988, Samuel Silverstone, 24 cells, 158 genes 0. 349573, Molly Silverstone, 12 cells, 82 genes 0. 369854, Ikroop Dickens, 14 cells, 97 genes 0. 438446, Leonardo Shakesphere, 19 cells, 144 genes Best in generation 3 is 0. 458743 score: 0. 458743, Akira Palestrina, 7 cells, 86 genes score: 0. 314851, Quentin Shakesphere, 8 cells, 61 genes score: 0. 295788 , Ella Joyce, 6 cells, 41 genes score: 0. 476386, Jane Palestrina, 8 cells, 53 genes score: 0. 133919, Ludwig Van Shakesphere, 3 cells, 62 genes score: 0. 321841, Jean Shakesphere, 11 cells, 97 genes score: 0. 337629, Rebecca Dickens, 9 cells, 63 genes score: 0. 405638, Amadeus Shakesphere, 24 cells, 151 genes Best in generation 4 is 0. 476386

Sound Examples (early results) Target Generation 1, best Generation 57, best

More to do. . • The algorithm is not yet complete – Logging not implemented – Evolution parameters need to auto adjusted – More Sound. Roots and Transformations needed.

Early Conclusions on Gene. Synth • The GA can produce meaningful results – If a target is supplied it “borrows” some of Its meaning and uses It to search. • The GA is useful for sound fitting and variation. – Because there is no mathematically defined optimum for this, there will be many ways to search for it. • Gas often find interesting niches and hot spots in the search space (local maxima, ) some of whichh may be interesting. • Generating a near perfect model has not been done yet – It will be a while before we can tell this, because the algorithm does not run fast enough yet.