In Depth: Immersive Audio
In this Article
One of the great pleasures of life is listening to an incredible mix of music on headphones or watching a film with immersive audio that just sucks you in. I love those binaural podcasts, also known as “First Person Immersive” (FPI) audio dramas, such as Calais 2037, Darkest Night, Deadly Manners. Immersive audio experiences can be found in the cinema, computer games, Interactive VR and 360° video, and even amusement park rides. Now it has been adopted with YouTube and Facebook 360 video.
First, we’re going to talk a bit about the terminology. Then, a few examples of immersive and 3D Audio. Many NLE’s and DAWs now include immersive audio tools, including DaVinci Resolve, Adobe Premiere Pro, and Avid Pro Tools. Build a better sounding VR or Immersive experience with tools you already have. These tools have features to up your immersive experience or VR game! In other words, learn to edit, place, and export 3D sound.
I’m going to define these words in the most basic of layman’s terms. Hooke Audio has a clear and informative article on the differences between these types of audio formats if you want something more detailed.
Mono: Monaural sound. Sound reproductions that appear to be coming from one position, or on a single channel, usually centered in the sound field.
Stereo: Also known as stereophonic sound. Sound recordings that simulate multidirectional sound.
Surround Sound: A system of placing sounds so they appear to surround the listener, using multiple audio channels. THX, Dolby, and DTS are known for surround sound. Surround Sound in Radio Drama by Eero Aro
3D Audio or Binaural Recording: Also known as VR Audio or Spatial Audio, where sounds are placed all around you. Binaural audio is recorded using two microphones to create a 3-dimensional soundscape for the listener. The idea is to reproduce an authentic real-life audio experience and fool your brain into thinking that you are immersed right in the scene. Binaural audio has been around for decades.
Ambisonic Audio: A 360 format that immerses a person in a field of sound, even above and below the listener. Below, we’ll get more into ambisonics.
Some Common Formats and Systems of Immersive Audio
Auro-3D: An immersive 3D audio format used in home theaters, cinemas, and sound over headphones. Learn more.
Dolby Atmos: A popular surround sound format, commonly used in films, but also in home theaters, mobile, and VR. It adds an overhead sound to the traditional surround sound.
MPEG-H 3D Audio: An MPEG codec that supports binaural and 3D formats. See more under Free Tools for Authoring Immersive Audio.
Sony 360 Reality Audio: Another popular format used in music.
Although there are a few formats out there, we’ll be focusing on the Ambisonic Audio B-Format, because it is the most commonly incorporated into software you may already use.
Ambisonic Audio was developed in the UK and has actually been around since the 1970s, and has gained popularity because of its wide use in VR and 360° video. The 4-channel B-Format mimics a full sphere of sound using regular audio channels. What’s interesting is that it doesn’t take a signal and send it to specific speakers like surround sound or stereo does. Instead, the channels work together to position the sound in 3D space.
Waves have a great explanation of Ambisonics B-Format and how to listen to mixes with Ambisonics B-Format.
More videos on Ambisonic Audio
- What are Ambisonics? – This video goes into the microphones that capture spatial audio.
- Ambisonic Audio Fundamentals (2020) | Simon Hutchinson – Simon Hutchinson gives a great explanation of Ambisonic Audio.
To pint out, make sure you put on headphones or have a surround sound or a speaker array set up to get the full effect. These are a variety of videos to give you a broad idea of how 3D audio can be used.
The Turning Forest
This is a real-time CG VR experience on Google Play from BBC production and it uses binaural audio. Download and play the game here. Below, the video is about the making of The Turning Forest. They started the game from the sound.
3D Sound WAR Virtual Reality Sound
This WW1 film short is not a 360 VR video but the 3D audio is really quite amazing.
Snowball – A VR 360° + Spatial Audio
Age of Sail
Age of Sail is a VR animated short film adventure set in 1900 and was featured at Google Spotlight Stories and an Oscar Nominated Animated Short 2018. It’s the story of a cranky, hopeless sailboat captain named William Avery, voiced by Ian McShane, who is on a mission to die adrift at sea in the North Atlantic. He rescues a young girl named Lara, voiced by Cathy Ang, who fell overboard from a cruise ship and with her help, and he realizes that he has a reason to live.
Age of Sail was produced with Chromosphere and Evil Eye Pictures. According to Animation Magazine, animators used Autodesk Maya, along with Google Spotlight’s proprietary game engine running it all in real-time. “The Spotlight Stories team spent a lot of time perfecting its audio engine to incorporate binaural sound, and account for the changes in the environment. When one of the two characters turns away from the viewer, their voice changes, and when the viewer turns her head, they get to hear the sound differently. And at one point, the viewer finds herself in the water, complete with the ability to put her head under the surface, which again completely changes the aural perception.” – Venice: The Story Behind ‘Age of Sail,’ Google’s Latest VR Short Film (EXCLUSIVE)
The VR version and the versions for mobile viewing let you look around the ship as the action takes place. Download Age of Sail at the following websites:
- Google Spotlight Stories and Chromosphere Raise the Bar for ‘Age of Sail’ – Animation World Network
- John Kahrs Enters the ‘Age of Sail’ with Cinematic New VR Short – Animation World Network
Ambisonics Audio 3D 360 Music Video – One More Chance
Microphones, headphones, and head trackers.
In post-production, helpful tools are a good set of headphones, obviously, and a Head Tracker!
Not to mention, you do need quality headphones to hear your 3D audio properly when you’re working on a project like this. Likewise, head tracker, as pictured above, is also extremely useful.
During NAB 2018, we stopped by the Waves booth and tried their spatial software with headphones and the Waves Nx Virtual Mix Room software and NX Headtracker. These headphones have a tracker on top, circled in yellow in the photo that changes the direction of the mix as you move your head. It was very cool. The example that we heard was not attached to any VR or 360 Video but was just a dance music track. It was stunning. We were at the Waves booth for over an hour chatting about plug-ins and mixing audio and the Waves guy handed us a head tracker to take home!
Below, watch a video with Nick Murray. He is a composer for film and TV who just got the Waves NX Headtracker. He shows you how it works.
The Abbey Road Studio 3 plugin brings the control room of Abbey Road Studio 3 to your headphones.
Produce and mix reliably wherever you are. Head tracking by the Nx Head Tracker enhances the plugin’s immersive realism.
Immersive Audio in Premiere Pro
Record & Edit Ambisonic Spatial Audio for 360 & VR180 in Premiere for Oculus Go, Facebook, YouTube
Hugh from CreatorUp walks you through the workflow for spatial audio. Of course, he’s a trip, but there is a lot of great information here.
Ambisonic Audio with 360 Video in Premiere Pro
Jason Levine talks about how to set up and publish a 360 video with 3D audio, specifically, from Premiere Pro to YouTube.
Ambisonic Audio for Youtube 360 Videos in Premiere Pro – Advanced Tutorial
- Support for VR Workflows, Considerations when using Ambisonics Audio
- Audio Mixing for VR: The Beginners Guide to Spatial Audio, 3D Sound and Ambisonics
Immersive Audio in DaVinci Resolve
DaVinci Resolve’s Fairlight audio includes built-in 3D audio support for working with spatial formats, including 5.1, 7.1, and even 22.2. The Resolve 16.2 release supports Dolby Atmos, Auro-3D, SMPTE ST2098, & MPEG-H.
To use MPEG-H 3D immersive surround sound audio bussing and monitoring workflows: Fairlight > Immersive > Generate MPEG-H Audio File.
Immersive Audio in Avid Pro Tools
VR Audio: Intro to Ambisonics in Pro Tools
There are a series of 4 Pro Tools Tutorials that explain how to edit and work with Ambisonics audio.
More from this series:
- VR Audio: Session Structure
- Tips & Tricks | Ambisonics and Facebook Spatial Workstation with Pro Tools – You can create 3D audio with the standard version Pro Tools HD.
Avid Online Learning — Creating Immersive @Dolby Atmos Music in Pro Tools
Steinberg Cubase Pro and Nuendo: VST AmbiDecoder Panel for Immersive Audio monitoring
In August 2020, Steinberg debuted the VST AmbiDecoder Panel, an AI personalized spatial audio design tool. It is included in Nuendo and Cubase for the monitoring of immersive audio.
VST AmbiDecoder Panel
The VST AmbiDecoder Panel converts Ambisonics audio for playback on headphones or with stereo or control room speakers for accuracy in monitoring. This is included with Steinberg Cubase and Nuendo. Below, is a video for Immersive.
With the increasing demand for content for VR and 360° videos, Nuendo 8.2 introduces a new Ambisonics bus system (up to third order) and supports most of the current VST 3 VR authoring tools available for DAWs.
Ambisonics is a technology that creates a spherical sound field. In opposite to traditional immersive sound formats, Ambisonics is not channel-based; it uses an encoded bundle of multiple audio streams to place sounds at any place within the sound sphere. The precision of positioning depends on the number of streams, and it is described as the first, second, or third order.
The implementation of the Ambisonics bus in Nuendo is just the first stage of our continuous strategy towards providing a complete solution for VR audio production out of the box. The rest of the necessary tools for such tasks, including an Ambisonics panner, a head tracking system, and a binauralizer, among others, will be implemented to Nuendo in a maintenance update later this year.
Create. Produce. Mix Repeat.
Cubase is one of the most powerful music creation software packages in the world. With its unrivaled range of flexible tools, you can create any kind of music quickly and intuitively. It comes packed with a wide range of virtual instruments, effects and thousands of sounds. Whether you’re a professional composer or a music production beginner, Cubase provides you with everything you need for turning your ideas into music.
Audio Ease 360Pan Suite
This is a video explaining Audio Ease’s 360Pan Suite and I think it does a great job of illustrating how 360° audio in video works.
The Audio Ease 360pan suite is a plug-in suite for Windows and macOS designed to deliver Ambisonics mixes for immersive audio: audio from all directions, intended for headphone reproduction, that stays where it is even when you turn your head.
You can do all your panning, distancing, and even mixing from within the video window of your DAW (Reaper or Pro Tools HD/Ultimate). There is no need to have any plug-in interfaces open while working.
Other Audio Tools for working in Immersive Audio
Post-production for Film, TV, and Games
Compliance, quality, and creativity – audio enhancement and problem-solving tools to help you produce outstanding audio quality on time, every time. For those working in Dolby Atmos, Auro 3D, and Ambisonic environments, a 3D immersive audio option is also available.
Natural reverbs for surround and immersive formats
Create immersive surround environments with unrivaled clarity using Stratus 3D. With up to 24-channel support, over 1,700 surround presets, and intelligent multichannel reverb controls, use Stratus 3D to get just the right amount of depth and dimension in any post-production job.
3D Surround Sound Libraries
If you want to add some ambiance and sound design to your immersive projects, without recording your own effects, you have options.
The Full Boom Library Experience In One Package
This is the ultimate collection for professional and high-quality sound design. All 58 original BOOM Library packs guarantee you the right sound at the right time for professional post-production, trailers, games, movies, TV shows, and anything else that needs a striking sound design.
Royalty-Free 3D Winter Ambiences
Whether you are looking to ease your audience into a blissful escapist nature documentary or raise the suspense in a bitterly cold finale, Seasons of Earth – Winter lends itself beautifully to any envisioned scene.
Our planet has a fascinating sonic spectrum, and every season, nature offers us an impressive and multi-faceted sound composition. We have recorded the most interesting and iconic ambiances the only way that truly reflects the sonic character of Earth as precisely as possible: Using the unparalleled Schoeps ORTF3D Surround Set. Harness these first-rate sounds to shape your projects in the most authentic way possible, from the ground up in 3D Surround or Stereo.
Earthly, Yet Alien
This library seemingly transports you to someplace far from Earth. However, these glorious sounds represent the primal geological processes of our beautiful planet.
Whether you are designing for daunting underground environments or films somewhere else in our solar system, draw from the rich sonic details in this collection, and fill your fictional realm with uniqueness and character.
GEOTHERMAL – 3D SURROUND AMBIENCES
Yellowstone National Park, this library transports the listener to someplace far away from Earth – although sounds represent the primal geological processes of our beautiful planet.
Whether you are designing for daunting underground environments or films somewhere else in our solar system, draw from the rich sonic details in this collection and fill your fictional realm with uniqueness and character – all in incredibly immersive 3D SURROUND!
Royalty-Free Room Tone Ambiences
The fourth entry of our famous sound effect series 3D SURROUND is now available: After ROOM TONES EUROPE comes ROOM TONES USA.
As you’d expect from a library labeled “room tones”, the sounds come without much action in the foreground to provide a very clear and detailed acoustic image of each room, giving you the perfect background noise layer to break the silence in your post-production project. The recorded rooms vary from hallways and warehouses over cafes, offices, basements, apartment rooms, laundry and boiler rooms to industrial rooms, and even an underground bunker. ROOM TONES USA covers all different scenarios you could possibly need, delivered fully surround with unprecedented precision, spatial impression, and clarity. Undoubtedly, this can be easily used in stereo productions as well. Either chose the angle yourself or use the stereo version (included in 3D SURROUND).
European City & Town Ambiences in Immersive 3D
Strengthen your productions with characteristic, intimate, and authentic European city ambiances including Daytimes & Seasons, Historical Places, Cultural Sites, Transportation & Traffic, People, Industrial, Abstract, Commercial, Natural Environments, Location Perspectives.
3D Surround Urban City Ambiences
This library, delivered in our brand-new 3D Surround format, is the perfect fit for your 3D urban ambient scenes. From calm downtown areas to busy crossroads and markets, it features both in- and outdoor subway areas, train stations, highways, parks, fountains, urban beaches, and much more.
Free Tools for Authoring Immersive Audio
iZotope: Free Practice Kit – Prepare a 360° Mix Session
Waves have a free download with a .zip file that will get you started quickly. Download a Pro Tools demo session with 360 audio, already laid out for you. Instructions are laid out for you. There are also links to two Ambisonics plugins as well (not free).
Facebook 360 Spatial Workstation
A free software suite for designing spatial audio for 360 video and cinematic VR. Learn more
MPEG-H Authoring Plug-in
Fraunhofer MPEG-H Authoring Suite (MAS) includes Open Source Software (OSS). Go to the end of this page to see if you qualify.
Spatial Audio and Ambisonics Freeware Tools – A long list of free options.
List of Ambisonic software – This list includes Free/Open Source Ambisonics Software.
Ambisonic.Info – a huge collection of information on Ambisonics
Envelop For Live
Free on GitHub is a collection of free tools for immersive audio for Ableton Live 10 Suite. Get Envelop for Live here.
Posted by Michele Yamazaki