Multimedia Technologies for Enriched Music Performance, Production, and Consumption

I nternet access, mobile devices, social networks, and automated multimedia technologies enable sophisticated information analysis and access. They have radically changed how people find entertainment, discover new interests, and generally express themselves online—seemingly without any physical or social barriers. Furthermore, thanks to the increasing affordability of sensing, storage, and sharing, information is taking on increasingly rich and hybrid multimedia forms. We now are frequently confronted with multimodal information streams, which are cooccurring in various social consumption settings. This phenomenon has also enabled opportunities in the music domain. In terms of music performance, musicians can exploit (live) analysis of and novel interaction mechanisms with musical data in multiple modalities, creating new forms of expression. For music production, sophisticated multimedia data analysis techniques can lead to more efficient and scalable workflows and richer and more accessible interfaces. In music consumption, music data containing embedded contextual and social information can lead to novel consumer experiences, stimulating greater music appreciation. This way, concerts turn into multimodal, multiperspective, and multilayered digital artifacts that can be easily explored, customized, personalized, (re)enjoyed, and shared among various types of users. Similar notions and opportunities hold for the consumption of general music recordings. The goal of this special issue was to gather state-of-the-art research on multimedia methods and technologies aimed at enriching music performance, production, and consumption. In recent years, researchers in the multimedia community have started looking at music as not just an audio-only phenomenon but as something that should be studied in multimedia contexts. Another aspect of interest to the multimedia community is that to become meaningful, music requires human listeners. As such, music encompasses both a multimodal representation and a human consumption context.