The Piano Music Companion

We present a system that we call ‘The Piano Music Companion’ and that is able to follow and understand (at least to some extent) a live piano performance. Within a few seconds this system can identify the piece that is being played, and the position within the piece. It then tracks the progress of the performer over time via a robust score following algorithm. The companion is useful in multiple ways, e.g., it can be used for piece identification, music visualisation, during piano rehearsal and for automatic page turning. 1 The Piano Music Companion The piano music companion is a versatile system that can be used by pianists and more widely by consumers of piano music, in various scenarios. It is able to identify, follow and understand live performances of classical piano music – at least to some extent. The companion has two important capabilities that we believe such a system must possess: (1) automatically identifying the piece it is listening to, and (2) following the progress of the performer(s) within the score over time. To support these two capabilities, the companion is provided with a database of sheet music in symbolic form, i.e., sequences of (noteon, pitch) pairs. Currently the database includes, amongst others, the complete solo piano works by Chopin and the complete Beethoven piano sonatas, and consists of roughly 1,000,000 notes in total (about 330 pieces). When listening to live music, the companion is able to identify the piece that is being played, and the position within the piece. It then tracks the progress of the performers over time, i.e., at any time the current position in the score is computed. Furthermore, it continuously re-evaluates its hypothesis and tries to match the current input stream to the complete database. Thus, it is able to follow any action of the musician, e.g., jumps to a different position or an entirely different piece – as long as the piece is part of the database. The system is tolerant to performance errors and slight variations, and is robust to tempo changes. These capabilities enable various application, of which four are described in Sec. 2 below.