Elizabeth Appelquist

Audio development in both digital games and movies has lagged behind the development of visuals for decades. Despite surround sound technologies like Ambisonics having been used since the 1970s, it has only recently been adopted into audio formats outside of the music domain. Ambisonics is a full-sphere surround sound format that can represent sound above and below the listener as well as in the horizontal plane. Unlike surround formats like Dolby Atmos, it does not carry speaker signals and instead contains a speaker-independent representation of a sound field, which can be decoded into any speaker setup. Therefore it is one of the most versatile sound formats and offers both the sound designer and the listener an incredible degree of flexibility. The first Ambisonic game was developed by Codemasters in 2007, called “Colin McRae: DiRT”, but the Ambisonic format was only released on PlayStation 3. Since then, Ambisonics has become more widespread across games and platforms, and has potential to become the new audio standard in video games.

This study investigated Ambisonics by developing and testing an Ambisonic game, which can be split into two main parts. The first focused on the game development and reasoning, 3D spatialisation and automation, and the result of the two audio mixes. The game, Pirate Ship Revenge, does not focus on audio as a method of gameplay, but instead uses the sound format to enhance the player’s experience, much like a standard video game. The second part of this study took the two versions of the game, a binaural mix and a stereo mix, and tested their ability to enhance player experience. Ethical approval was given for participants to play each game and compare their experience through an online form.

The final version of the game had a range of sound effects and ambiences that varied and changed as the game progressed. The main ambience consisted of a collection of sounds automated around the listener, including seagulls, ship creaks and the sea. This ambience gave a fantastic opportunity to explore 3D spatialisation panning and automation due to the amount of movement required for each of these sounds, and was the most notable difference between the binaural and stereo mixes. The seagulls were played in a random order combined with randomised movement around the emitting game object through automation. The sea was automated to sound from below the player. The ship creaks were automated to creak from left to right followed by right to left and were attached to the ship. The sound of the sea changed when the ship started moving, and also contained rattling and thrashing of the sails which was automated to sound from above the player.

There were two additional game objects which emitted interesting soundscapes for the player to listen to. The first being the hatch in the middle of the ship, and the second being the cabin doors. These were included to give the player a dynamic sound environment to experience and explore which were not part of the main ambience. These soundscapes also had no player interaction, which meant the sounds played with no prompt from the player, unlike any game objects with attached sound effects. The cabin doors emitted a conversation between two pirates counting and describing the gold they had stolen. The sound had the same filters and effects as the hatch, but the low-pass filter attached to the cabin doors’ attenuation was never zero at the closest distance, simulating the sound travelling through the door. The hatch was the only soundscape to change once the ship started moving – from a lively party atmosphere to heavy lifting and movement from the pirates at work.

Sound effects are a key part of any video game to acknowledge player interaction. For this game, the main game objects to make noise were the barrels, the treasure chest and the wheel, when picked up or dropped. The player also emitted the sound of footsteps to indicate walking, which would change depending on whether the player was walking on deck or up and down stairs. Each barrel had a distinct sound depending on whether it was big, small or had rum in it, and the treasure chest followed suit with a distinct noise for being picked up, dropped and thrown overboard. The wheel was the only game object to have an additional sound for when it attached itself to the helm. These player interactions were encouraged by the game objectives in the top left corner of the screen, for example, “throw loot overboard”, so that the player explored the many sounds in the game. It also forced the player to move a great deal, meaning all the 3D spatialisation would have been more noticeable, and therefore aiding the participants’ comparisons between the binaural mix and the stereo mix.

Participants were told only the names of the two games and not which was which. For clarity, Game X was the stereo mix, and Game Y was the binaural mix. When comparing the two versions of the game, a large majority of participants were in favour of the binaural mix of the game. 85% of participants (17) found it both more immersive and more enjoyable than the stereo mix. Two participants (10%) could not tell the difference between the binaural and stereo mixes, but guessed correctly when asked to tell them apart, indicating that whilst not consciously recognising the difference, both guessing correctly indicates a subliminal appreciation of the difference. Only two participants (10%) guessed this wrong and mistook the binaural mix for the stereo mix. Therefore their answers for game immersion and game preference were answered as if the games were the other way around. One participant claimed it was because Game Y felt “a little emptier”, while the other thought Game Y had the mantra, “if the sound happened lets make it loud”. Only one participant preferred, and felt more immersed by, the stereo mix knowingly. They stated their reasoning as “you could hear the voices fade in and out which was cool. However, Game Y seemed more clear where the sound was coming from”. Consistently, participants preferred the game they found most immersive. Of the two participants who could not tell the difference, neither had hearing problems. Interestingly, their musical or audio experience did not influence their ability to distinguish between the two games, but both participants regularly play video games. It is likely both participants could not tell the difference because they are are more used to the gaming experience and would not have experienced the same level of immersion as a non-player.

Despite the vast majority of participants preferring, and feeling more immersed by the binaural mix, the intensity of immersion varied dramatically. While some felt the binaural mix made it more “real”, “authentic” and “involved” because of its directionality, others believed they would not have noticed the difference had they not been looking for it. One participant stated “with [Game] Y it was much easier to forget about the fact that it’s a game with audio clips playing and just accept the ambience as ‘real’ and move on. Also, the sound mix in Game X often seemed to mush together in a way that made it very difficult to pick out relevant noises, and the volume changes from distance felt oddly artificial”. Another stated “without knowing there was a difference, I didn’t notice any difference. So I didn’t think one was more immersive than the other”. Despite the varying level of immersion, participants still preferred the binaural mix. When asked for any additional comments, a few participants said they had gone back and played both games again and enjoyed them more, after realising the difference between them at the end of the survey. Other participants used this opportunity for additional comments to convey how impressive they found the difference in sound between the games – “very impressive how you can manipulate the sound in such a way in which the two games feel different despite the narrative and the visuals being the exact same”.

The results from this survey suggest that players can successfully tell the difference between a binaural game and a stereo game, and felt more immersed by, and thus preferred, the binaural game. Participants felt the binaural mix’s directionality aided their immersion and added to the realistic nature of the game. The majority of participants who took part felt sound is an important part in games and proves that the role Ambisonics has to play with game sound should be examined closer. It has many exciting current uses and many promising future opportunities, possibly in a learning environment to add further realism to simulations.

This study has its limitations, and although participants were overall more immersed by the binaural mix, this level of immersion clearly varied and was not quantified. One possible explanation could be due to the individual’s Head-Related Transfer Functions (HRTFs), which characterises how an ear receives a sound from a point in space. As head and ear shapes differ, so does their HRTF. The Auro Headphone effect will have filtered the sound to create a standardised HRTF, which will have been experienced differently by each participant. It is possible that those with an HRTF closest to this standardised HRTF felt most immersed by the binaural game, which could influence further research. This could involve measuring this level of immersion in each version of the game similar to studies done by Gormanley and Manuel et. al or measuring participants’ HRTF and testing its immersive properties.

Furthermore, there was no possibility of keeping participants’ listening environments and technological restrictions consistent due to the remote nature of the study. Another future investigation might invite participants to use the same computer and speaker setup, and even use a surround speaker array, such as the Sound Lab at the School of Simulation and Visualisation, instead of headphones. The game itself did not require sound to be able to progress, as most games are not dependent on audio for their gameplay. Further research could involve repeating this study, still comparing spatialised sound and stereo, with an audio game or an audio-only game, whereby the game would be unplayable without sound. This would determine if a forced direct focus on the game audio influences a player’s immersion and enjoyment of the game.

T. Farkaˇs, “Binaural and ambisonic sound as the future standard of digital games”, Acta Ludologica 1(2) (2018).

F. Zotter and M. Frank, Ambisonics – a practical 3d audio theory for recording, studio production, sound reinforcement, and virtual reality, Vol. 19 (Springer Topics in Signal Processing, Gewerbestrasse 11, 6330 Cham, Switzerland, 2019).

S. Gormanley, “Audio immersion in games – a case study using an online game with background music and sound effects”, The Computer Games Journal 2(2) (2013).

D. Manuel, D. Moore, and V. Charissis, “An investigation into immersion in games through motion control and stereo audio reproduction”, AM ’12: Proceedings of the 7th Audio Mostly Conference: A Conference on Interaction with Sound, 124–129 (2012).