Jonathan Kilpatrick Undergraduate Dissertation 2014/15

A virtual drum kit controlled by LEAP Motion

Supervised by G.Brown

Abstract

The LEAP motion sensor is a motion tracking infrared sensor, able to track the user's finger movements with excellent precision. The goal of this project is to build a real-time drum synthesiser controlled by the LEAP motion device, allowing the user to produce a percussion track simply by tapping their fingers in the air. The project will create virtual Tabla drums, the percussion instrument used in Indian folk music, a drum with a strong sense of pitch and many different playing techniques and strokes. Physical modelling of drum sounds will also be investigated allowing the user considerable control over the tone, dynamics, pitch and rhythm of the drums, depending on how hard, and which part of the drum is hit.

The system includes a robust gesture recognition system based on the position and velocity of the finger movements, with a visually appealing display of the fingers in relation to the drums. The project is developed in the Java programming language, using Processing for graphical display, and the JSyn audio library to produce sound. Achievements include a working gesture recognition system to interact with the LEAP motion device, and a real-time Banded waveguide synthesiser, tuned with the use of a genetic algorithm.