Game/Music Interaction - An Aural Interface for Immersive Interactive Environments

Game music has the potential to be much more than a passive element of the background. The music can and should affect the game play. The game play can and should affect the music. The player’s actions can and should influence the direction and evolution of the music. By tightly linking game play and music, the player becomes much more immersed in the experience, and new creative possibilities abound for the developer. This paper presents a framework for linking game play to music by analysing music on a perceptual and a physical level. Real-time processing is achieved by using the GPU as an APU. The usefulness of the framework is demonstrated by two examples of game play synchronised with auditory perceptions. It is our hope that this paper will enable and stimulate the reader to create musically interactive games, and discovering entirely new ideas in this field.