Implementations of the Leap Motion in sound synthesis, effects modulation and assistive performance tools

The Leap Motion opens new possibilities for mapping the various degrees of motion of the human hand with musical expression. The Leap Motion is a computer peripheral released in mid 2013 that uses IR cameras to track hand and finger location with unprecedented accuracy. In this paper, we explore implementations of the device in sound synthesis and effects control. The device is interfaced with Max/MSP to provide motion and finger-based control over multiple parameters in a software synthesizer. Next, we implement a 5-grain granular synthesizer where users trigger individual grains by depressing their fingers in mid-air. While triggering grains, users can simultaneously move their hands to dynamically modulate grain length and scrub the sample. The benefits and limitations are discussed in light of recent compositions and performances. The Leap Motion is also used to spatialize the synthesized sound produced from a 6-channel hemispherical speaker. Applications to music composition and music therapy are discussed.