Real Time Walkthrough Auralization the first year B

Real Time Walkthrough Auralization - the first year B. -I. Dalenbäck CATT M. Strömberg Valeo Graphics Gothenburg Sweden • from static to dynamic auralization • properties and limitations • model and receiver grid examples • current and future options • applications and summary 1

From static to dynamic auralization: 1/6 • traditionally, starting around 20 years ago, auralizations have been static: fixed listening positions with fixed head directions. • a single static, typically binaural, room impulse response (FIR) is convolved with anechoic sound • schematic representation of a static FIR (one head direction): direct early L late time R 2

From static to dynamic auralization: 2/6 • a head-tracked binaural early part FIR with a static late part, medium calculation time, fairly memory consuming: direct early late L time R Many early part FIRs corresponding to many head directions A single late part FIR corresponding to one head direction 3

From static to dynamic auralization: 3/6 • a head-tracked full-length binaural FIR, long calculation, very memory consuming: direct early L late time R Many full-length FIRs corresponding to many head directions, still just a fixed position 4

From static to dynamic auralization: 4/6 • the solution: full-length B-format FIRs, short calculation, not memory consuming: direct early W X Y late time Binaural decode L R Z A single full-length B-format FIR, rotation performed afterwards. Each head direction created when needed 5

From static to dynamic auralization: 5/6 CATT-Walker™, putting it all together with many positions, rotating and interpolating while convolving Pre-processing 6

From static to dynamic auralization: 6/6 Real time processing details 7

Properties and limitations • the real time part of the process is independent of room complexity (3 sec church “=“ 3 sec shoebox) • the prediction and post-processing methods are exactly the same as for static auralization • requires a high receiver density where the IR is expected to change fast with movement or head direction 8

Model and receiver grid examples : 1/4 A church walkthrough, 80 receivers, plan view 9

Model and receiver grid examples : 2/4 The church walkthrough, 3 D view 10

Model and receiver grid examples : 3/4 The church walkthrough, CATT-Walker™ view 11

Model and receiver grid examples : 4/4 A smaller room walkthrough, 3 D model view 12

Current options : 1/2 · multiple source simulation at no extra CPU cost (assumes that sound input is common for all sources such as in a PA system) · choice of HRTFs for the binaural decode · choice of headphone eq. for the binaural decode · variable walking speed · optional TCP/IP control via the Walker Steer API · optional trade-off for use with slower PCs (latency and/or horizontal only i. e. based on WXY) 13

Current options : 2/2 Optional grid and WXYZ FIR view 14

Future options : 1/2 · detailed models with textured graphics imported from programs such as 3 ds max · head-tracking at no extra CPU cost (not crucial in front of a PC screen) · multiple independent sources (higher CPU demand) · Doppler effects (difficult with FIR interpolation, not crucial for room acoustics) · direct B-format output for external decoding to any loudspeaker array (lower CPU demand) 15

Future options : 2/2 · ambisonic output for direct loudspeaker replay (lower CPU demand) · use of measured instead of predicted B-format FIRs (can be measured by a Soundfield microphone) · use of 2 nd order B-format FIRs (higher CPU demand, not crucial for a binaural down-mix) 16

Applications · everyday use (processing only longer due to a higher number of receivers) · presentations to clients · presentations in architecture competitions · research projects exploring the possibility to control via TCP/IP. Two example EC-projects: · “POEMS” at Chalmers University, Gothenburg · “Wayfinding” at LIMSI, Paris 17

Summary · a technique for real time walkthrough auralization has been described: · based on B-format FIRs and binaural downmix · is in itself general and can as well be based on measured responses · no special shortcuts made for the real time option · future improvements of prediction and auralization methods will directly carry over to the walkthrough auralization 18