Introduction to Synthesis for Interactive Applications Leonard J

Introduction to Synthesis for Interactive Applications Leonard J. Paul Lotus Audio Video Game Audio. com

Overview Additive, Subtractive & FM Speech Synthesis Spectral Resynthesis Sampling Modal Synthesis Granulation Physical Modelling

Synthesis for 7 th Gen Video Games Generate audio in real-time based on rules Has been used in games since the beginning Parametric, generative & real-time (ie. flexible) Can combine methods Allow flexibility of layers instead of stereo bounce

Basic Synthesis Additive & Subtractive Synthesis, Frequency Modulation (FM), Wavetable Synthesis Good for sci-fi sounds, “sweeteners”, electronic music and retro sound design Can also be creatively used to simulate real-world sounds Crash Engine Explosion Ouch

Additive Synthesis Add simple (sine, triangle. . . ) waves together

Subtractive Synthesis Take a complex waveform and filter it That classic “analogue” sound

Frequency Modulation Synthesis Modulate frequency of one wave by another at an audio rate Yamaha DX 7, PC Adlib or Sega Genesis

Sampling Wavetable synthesis is a short looping sample Multi-sampling is a collection of samples Good generality and control but RAM intensive and difficult to parametrise Pervasive in games today Possibly use granulation or other processing techniques to “bend” samples Examples include Kontakt, Gigastudio. . .

Speech Synthesis A collection of methods: Concatenative, Linear Predictive Coding(LPC), Formant Synthesis. . . Current research examines dialects and accents Vocaloid product utilises speech “fonts” to base synthesis in an existing voice style

Spectral Resynthesis Examine Short Time Fourier transform (STFT) and resynthesize with different parameters Allows for morphing, time warping and pitch shifting Often has difficulty preserving transients

Physical Modeling Model sound production and propagation Difficult as each model is specific Good for metal, instruments, swords and more

Precomputed Acoustic Transfer Optimization of almost 7000 x over brute force Find out radiant audio from modes Good for plastic and metal objects

Modal Synthesis Examine the modes or resonances of a material and model them Good for wood, plastic and metal impacts

Granulation Segment a sound into grains and reconstruct Transients lost due to enveloping and mixing Presegment to preserve transients Can spacialise grains in surround

Concatenative Synthesis Uses grains of one sound to approximate the original sound, like an “audio mosaic” Reconstructs sample using a different set of grains and metrics to find close grain matches

Related Topics Parameter control From physics, massage inputs, complex data DSP Effects 3 D waveguide for reverb, wavetracing. . . Optimisation Large issue, try precomputing

Links Perry Cook - http: //www. cs. princeton. edu/~prc/ Curtis Roads - http: //clang. mat. ucsb. edu PAT - http: //graphics. cmu. edu/projects/pat/ http: //www. procedural-audio. com

Questions ? info { at } Video. Game. Audio. com