AES Fall Online Convention

In the general practice of immersive audio recording, there is a focus on how to capture the direct sound source. However, a sound source’s complex radiation properties help define that source in three-dimensional spaces. In this research, the idea of a “holographic sound recording” or HSR is explored. We choose the term “holographic” due to its uncanny ability to create a rendering of a real 3D sound source in space, similar to holographic visual experiences.

HSRs can be defined as the capturing of complex radiation characteristics of sound sources with the intention of being played back in either a virtual environment with six-degrees of freedom or in real life through a multi-directional coincidental speaker array. To research techniques on creating sonic holographic reproductions, two recording sessions were conducted at NYU in the summer of 2021. Through documenting and reflecting on the miking, mixing, and spatialization of the audio objects in Unity with Google Resonance, experimental realizations were made in search of best practices to consider when creating HSRs.

Concepts such as acoustical points of interest, adequate number of microphones, pickup patterns and angles will be explored, as well as capturing room tone and the benefits of player isolation. These aspects are introduced to create a holographic miking system called the Multi-Channel Pyramid Array (MPA) as a starting point for users who would like to create a holographic reproduction of any instrument. The MPA can theoretically be fine-tuned and customized based on the instrument and/or to the user’s desired results.