Optical Music Recognition and Playback on Mobile Devices

The recognition of printed sheet music has been studied since the late 1960s. However, current applications often require careful scanning of musical scores, making it inconvenient for average music enthusiasts. With the advances in mobile devices and network connectivity, we develop an optical music recognition system to remotely interpret printed sheet music captured by an Android phone and present the music through MIDI playback on the device. By applying various image segmentation, filtering, morphological processing, and feature detection techniques, the application is able to identify the pitch of the notes by spatial position, form chords based on spatial alignment, and generate corresponding music file through MIDI table lookup. This application can be used by music learners in practicing unfamiliar pieces of music. It can also be deployed as a software extension to the control unit of player pianos currently in the market. Keywords-optical music recognition; mobile image processing;