VR has the power to transport us to new worlds, but if designers get even one thing wrong, the whole magic spell disappears and we're in a video game again. Convincingly-realistic sound is part of that illusion. Valve has unveiled a new tool that developers can use to help make the magic happen: the Steam Audio SDK, available for free and usable in both VR and regular games.
When we play games on monitors and televisions, stereo and surround sound works well enough. The two-dimensional imagery is accompanied by what's essentially two-dimensional sound. The aural location may not match where it's visually coming from, but it's close enough, and our brains can bridge the gap. When we enter VR though, things become more complicated. The world before us is no longer two-dimensional, nor is it limited to the constraints of a screen. Humans are pretty good at being able to locate where sounds come from based on how they hit our ears, and a VR environment begets pinpoint-accurate audio. The Steam Audio SDK promises to help with that.
The SDK offers developers a host of features like HRTF, physics-based propagation with occlusion and reflection modelling, and and baked-in sound propagation. Since the Steam Audio SDK is intended mainly for VR, there's rotational and positional tracking on tap, too.
Physics-based reverb promises to let creators bounce sound around the environment to more clearly establish a relationship between the player and the source of the sound. Occlusion helps with altering sounds as they're heard through the other side of obstacles. The most important element, though, is probably binaural rendering. When hits our ears, it reaches each ear at a very-slightly-different time. That's a lot of what helps us locate a sound's source, and it's something we humans do intuitively all the time. Valve is promising the SDK will offer HTRF-based binaural audio with "very low CPU overhead," to the point where hundreds or thousands of audio sources could be handled by a single CPU core.
The SDK is currently available only for the Unity engine and as a C API, but it's coming to Unreal Engine and the fmod and wwise middleware. The Steam Audio package is available for free for all developers to use in any projects without royalties.