The Aesthetics, History and Future Challenges of Interconnected Music Networks

This paper presents an overview of the field of Interconnected Musical Networks – live performance systems that allow players to influence, share, and shape each other’s music in real-time. Informed by social and biological systems , these networks are designed to allow a group of performers to interdependently collaborate in creating dynamic and evolving musical compositions. The paper starts with a discussion of the motivations and aesthetics behind this unique form of social artistic expression. It then goes on to describe a number of historical and technological landmarks that led the way to the development of the field, from the early experimentations with the radio transistor as a musical instrument, through the introduction of digital technology and networked PCs, to the current proliferation in Internet music research. This review leads to the formulation of a number of future prospects and challenges which address the introduction of Interconnected Musical Networks (IMNs) to wider audiences, novices, and children. The paper concludes with a presentation of two IMN projects that were developed in the Hyperinstruments group at MIT Media Laboratory, which address these future challenges. 1 Concepts and Aesthetics Music performance is an interdependent art form. Musicians’ real-time gestures are constantly influenced by the music they hear, which is reciprocally influenced by their own actions. This interdependency is true not only in group playing but for soloists as well, for example, a violinist who is listening to the music she is playing and constantly modifying her actions with correlation to the auditory feedback stream. In group playing, however, the interdependent effect bears unique social consequences. Rudolf Rasch (1978) shows how group synchronization has a direct influence on individual players’ isochronization. Comparing onset times of ensemble performance (played to a multitrack recorder in an anechoic chamber) Rasch found a number of different social tendencies such as the formulation of leaders and followers (in milliseconds) or the effect of group synchronization on individual players’ dynamics and timing. Other models of group performance show different manifestations of interdependency. Standard jazz improvisation features interdependent routines such as call and response, propagating motives, supporting and contrasting dialogs, and a higher level of leader/follower dynamics. Non-western music presents its own variations of group interdependency, such as in the case of Gamelan music which is based the concept of heterophony the simultaneous performances of melodic variations on the same tune (countermelodies) or Persian art music, where instrumentalists are expected to vary the singers' improvised lines in real time. Fig 1. Acoustic group interdependency can be demonstrated in Gamelan music performance. Although the acoustic interdependent models provide an infrastructure for a variety of approaches for interconnections among players, they do not allow for actual manipulation and control of each other’s explicit musical voice. Only by constructing electronic (or mechanical) communication channels among players can participants take an active role in determining and influencing, not only their own musical output, but also their peers’. For example, consider a player who while controlling the pitch of his own instrument also continuously manipulates his peer’s instrument timbre. This manipulation will probably lead the second player to modify her play gestures in accordance with the new timbre that she received from her peer. Her modified gestures can then be captured and transmitted to a third player and influence his music playing in a reciprocal loop. Another example is a network that allows players to share their musical motifs with other members in the group. By sending a motif to a co-player who can transform it and send it back to the group, participants can combine their musical ideas and tendencies into a constantly evolving collaborative musical product. 1.1 The biological metaphors The shape of the composition in IMNs grows from the topology of the network and its interconnections with the performers. Such a processdriven environment, which responds to input from individuals in a reciprocal loop, can be likened to a musical “ecosystem.” In this metaphor, the network serves as a habitat that supports its inhabitants (players) through a topology of interconnections and mutual responses which can, when successful, lead to new breeds of musical life forms. Such IMN ecosystems differ from other closed process based musical networks (for example, modular analog synthesizers in which periodic behavior determine the dynamic nature of the musical output) in the significant role they provide to the real-time input from a society live performers. An examination of such systems calls for disciplines such as system dynamics, which looks at complex interdependent natural and social systems and tries to explain them by using computerized simulations. Another biological metaphor that can help illustrating the experience of participating in IMNs is the one of “gene mixing,” which is derived from the penetrative quality of diffusing and influencing each other’s music. The network, therefore, can be seen as a (pro)creative environment that allows a group of musician “parents” to give birth to their musical crossbred offspring. The members of “The League of Automatic Music Composers” – a computer -musicnetwork group describe how a coherent and vital musical entity can emerge from interdependency and feedback in one of their compositions: “There are moments of tuned correspondence where the voices seemed to listen to each other; at other times they appear to be independent. There are also instances of odd grandeur...when the elements of the network are not connected the music sounds like three completely independent processes, but when they are interconnected the music seems to present a “mindlike” aspect.” (Bischoff, Gold, and Horton 1978.) 1. 2 Coherency vs. immersion But achieving such a life-like effect requires the maintenance of a delicate balance. On one hand, the low-level scheme of the network should be kept comprehensible and intelligible so that players and audiences would be able to participate and follow the music in a meaningful and coherent manner. On the other hand, one of the exciting promises of IMNs is to provide participants with an interconnected immersive whole which can grow to be bigger than its parts, where low-level rationalization of rules and algorithms is counterproductive to the social and artistic experience. An effective IMN would, therefore, be able to help facilitating interdependent connections so that the group members would smoothly transfer between the two perception modes – the analytical low -level coherency and the more abstract and high-level immersion. Such interdependent dynamics, which is not possible with any acoustic means, would let participants to complement and enrich each other without losing control of their personal contributions. The network can allow, for example, for a soloist to guide his collaborators with a simple interdependent touch towards a musical idea that he is interested in, to change a supporting voice into a contrasting one so a desired musical idea will become clearer, to shape a peer’s accompaniment line so it would lead towards a new direction when the current one is exhausted, to send a motif to another player who would manipulate it and send his variation back to the group, to have a musical response accentuated by the player who sent the original call, to plant a musical “seed” that would be picked up by the group in various manners, etc. would be especially effective if it can allow players to 1.3 Music as a social ritual A fundamental aesthetic concept in IMNs is the computer’s role as a supporter and enhancer of live musical interaction with its surprise, immediacy, and flexibility. The system should be able to enrich the interpersonal interactions through its control and manipulation algorithms and to stir the musical output into unpredictable directions, leading to an experience that is based on evolving and dynamic social contexts. An effective IMN would therefore promote the interpersonal connections by encouraging participants to respond and react to these evolving musical behaviors in a social manner of mutual influence and response. This unique form of live performance can, therefore, enhance the inherent social attributes of music making that are usually obscured in many other forms of music technology practices. Home studios, sequencers, sound generators and other technological innovations can lead to a privet and isolated practice of music making. IMNs, on the other hand, bear the promise of brining back music performance to its social context and to its ritual roots. Interconnected performance, as opposed to common utilizations of technology in music, can provide a direct connection to the routs of music as exciting and immersive social ritual. 2 Historical Landmarks The concept of IMNs can be seen as a descendent of the tension that emerged in the midst of the 20th centaury between the radicalization of musical structure and composer’s control, practiced mainly by “avant-garde” and “post-serialist” composers such as Stockhausen and Boulez on one hand, and the escape from structure towards “Process Music” as was explored mostly by American experimentalist suc h as Cage and Reich. As opposed to the European movements that emphasized the composer’s control over almost every aspect of the composition, “Process Music” came from the belief that music can be a procedural and emergent art form and that there are many ways of handling form other than constructing structures in different sizes. In such procedural process-based music, the composer sacrifices certain aspects of direct control in order to create an evolving context by allowing rules (in closed system) and performers (in open ones) to determine and shape the nature of music. John Cage addressed this tension referring to his own experience: “I was to move from structure to process, from music as an object having parts to music without beginning, middle or end, music as weather.” (Cage 1961) The use of technology in IMNs pushes the tension between Structure and Process music further into an experience where predetermined rules and instructions combined with improvised interdependent group interactions leads to evolving musical behaviors, giving a new meaning to Cage’s exploration of unpredictability, chance determination processes, accidents, and contextual music emergent. In particular I see three major technological innovations which helped making such interconnected musical behaviors possible. These are the transistor radio, the personal computer, and the Internet. When these technologies became widespread and commercially available they inspired musicians who were looking for new ways to expand the vocabulary of socio-musical expression. I will base my historical review on these three technological milestones, starting with John Cage and his early 1950s’ experimentations with the transistor radio as a musical instrument that provides crude interdependent musical interactions. I continue with the “League of Automatic Music Composers” and their offspring group “The Hub,” which utilized the personal computer to create the first programmable digital IMN, and end with an overview of recent Internet music research, which focuses on scaling musical networks up to a large number of participants with a variety of musical backgrounds, while providing a wide range of interconnectivity models. 2.1 John Cage and the Transistor Radio – Technology for Interdependency John Cage was one of the firsts to take notice of the expressive potential that lies in using technology to enhance acoustic group interdependency by treating the then recently invented commercial transistor radio as a musical instrument that can be used to provide a sonic medium for interdependent procedures, rules, and processes. Cage’s compositions for transistor radios allowed, for the first time, for an external entity (audio steams from a set of radio stations) to generate and support evolving and dynamic musical contexts, providing a first crude glimpse at the concept of decentralized “musical ecosystems.” Cage’s 1951 Imaginary Landscape No. 4 for twelve radios played by twenty-four performers can be possibly considered as the first electronic IMN. The composition score indicates the exact tuning and volume settings for each performer but with no foreknowledge of what might be broadcast at any specific time, or whether a station even exists at any given dial setting. Inspired by the Chinese book of oracles, the I Ching, Cage demonstrated his fascination with chance operation, allowing players to control only partial aspects of the composition, while technology, chance, and other players determined the actual audible content. The role of Cage as a composer was narrowed down to setting the highlevel blueprint of dial setting instructions. The interdependency in the piece was manifested in two planes: First, there were the interdependent interactions between the players and the network of radio stations that provided unknown and dynamic music content. But the system also supported intraplayer interdependencies since for every frequencydial player there was a volume-dial player who could have manipulated the final output gain, controlling a full continuum from complete muting to maximum volume boosting. The volume-dial player, therefore, had a significant impact on her peer’s musical output, as she could control anything from rendering his actions inaudible, through blending them smoothly in the mix, to boosting them up as a screaming solo. Cage continued to experiment with interconnected compositions for radio broadcast in Speech (1955) for five radios and a news reader and Music Walk (1958) for one or more pianists, radios and phonographs. Addressing the biological metaphor, he referred to these compositions, stating that his goal was “to affirm this life, not to bring order out of chaos... but simply to wake up to the very life we’re living, which is so excellent once one let it... act of its own accord” (Cage 1961.) The explorations of the transistor radio as an infrastructure for interdependency opened the door for other explorations of electronic interdependency, which were not necessarily based on external sound production. In Cartridge Music (1960), for example, Cage made his first attempt at an IMN that is focused on tactile generation of sounds and intra -player amplification-based interdependencies. Here, players were instructed to pluck small objects (such as toothpicks or pins) that have been put into a gramophone cartridge, and to hit larger objects (such as chairs) that were amplified with contact microphones. The simple intra-player interdependency was generated due to other players who controlled the amplifiers’ volume knob, leading, again, to a wide range of output from muting to soloing. On Cartridge Music Cage remarked: “I had been concerned with composition which was indeterminate of its performance; but, in this instance, performance is made indeterminate of itself.” 2.2 The League, the Hub and the Personal Computer – The Digital Advantage Although revolutionary, the level of interdependency in Cages’ early experimentations had been constrained by the crude nature of the technology, where in effect, the only possible direct interpersonal connections were limited to coarse gain manipulations. This problem was addressed by the second technological milestone – the commercialization of the personal computer, such as the 1976 commodore KIM -1, which allowed for fine tuned and configurable network topologies. The League of Automatic Music Composers, a group of musicians from Oakland California was one of the firsts to use a number of KIM-1s to write interdependent computer compositions. By networking their computers, each composition could send and receive data from the other compositions, and for the first time to create programmable and detailed musical interconnections. The “League” named their new genre of musical performance “Network Computer Music.” In their 1978 performance in Berkeley California, for example, the group set up a 3-node network, mapping frequencies from one computer to generate notes in another, or mapping intervals from one composition to control rests and rhythmic patterns in another. The League continued to work until 1986 when it evolved into an offspring group, “The Hub,” which employed more accurate communication schemes by using the Midi protocol. The group also experimented with more hierarchical systems, such as in Waxlips (1991), where a “lead player” was sending signals to initiate new sections and to jump -start processes by “spraying” the network with requests for note messages, etc. The Hub also expanded their explorations to other areas such as remote collaboration and audience participation. These, however, were less successful, according to group member’s testimony. In their first 1985 remote networking effort, the group was divided into two sites and communicated via phone lines. The experiment turned ineffectual mainly because of technical problems that impaired the flow of the performance. Another less than perfect remote experiment was HubRenga (1989) in which the general public was able to interdependently participate in the composition through the Internet. Scot Gresham -Lancaster, a Hub member reflects: “The varying range of taste and innate talent made for a pastiche that lacked fitness and cohesion, and despite the best intentions of the contributors, the results were mixed” (Gresham-Lancaster 1998). Regarding their last remote interaction effort, using IP based OpenSoundControl for Max, Gresham Lancaster reasons: “the technology was so complex that we were unable to read a satisfactory point of expressivity.” The League’s and The Hub’s Network Computer Music contributed significantly to the field of IMNs by introducing the computer as a versatile and resourceful partner for interconnected group interaction. They were, however, less successful in supporting large scale systems for novices and wide ranged general public, challenges that were more successfully addressed by the Internet. Fig 2. “The League of Automatic Music Composers” introducing electronic group interdependency. 2.3 The Internet – Various Levels of Interconnectivity In recent years there has been an increasing interest in Internet based musical systems for multiplayer interaction and collaboration. The different approaches that were taken by composers and researchers vary in the musical activities they offer, the number of participants, the musical skills that are required, the level of hierarchy and real-time, etc. In this section, however, I will map the field of Internet IMNs based what I see as the central innovative concept of the medium – the level of interconnectivity among players and the role of the computer in enhancing these interdependent social relations. Based on these criteria I have identified four different approaches and named them “The Server,” “The Bridge,” “The Shaper,” and “The Construction Kit.” They are explained below. The Server Approach this simple approach uses the network merely as a means to send musical data to disconnected participants and does not take advantage of the opportunity to interconnect and communicate between players. Participants in such a server/client configuration cannot listen to, or interact with their peers and the musical activities are limited to the communication between each player and the central system. A typical example for the Server approach is the Sound Pool web application, which is part of the interactive piece “Cathedral” by William Duckworth (De Ritis 1998, Duckworth 1999). Here, a Beatnik based java applet allows individual players to trigger sounds by “accidentally or randomly” clicking on hidden nodes on the screen. The interaction occurs independently in each player’s browser so that “each user can create his or her own unique experience.” Since there are no connections between participants, the system can support any number of users. In particular, the application addresses “passive audience” and tries to “bring audience closer to the actual creation and performance of music.” The original sounds in the piece were composed by Duckworth but users can contribute their own sounds to be mixed in. Still, participants can only listen to their own creation, which significantly limits the sense of collaboration. The Bridge Approach The motivation behind the Bridge approach is to connect between distanced players so that they could play and improvise as if they were in the same space. Unlike the Server approach, musical collaboration can occur in such networks since participants can listen and respond to each other while playing. However, the role of the network in this approach is not to enhance and enrich collaboration, but to provide a technical solution for imitating traditional group collaboration. Aspects of bandwidth, simultaneity, synchronization, impact on host computer, and scalability are some of the challenges that are usually addressed in this approach. A characteristic example of the Bridge approach is the “Distribute Musical Rehearsal” project (Konstantas 1997), which focused on remote conducting. With the help of video streaming and a 3D sound system, an ensemble of six players in Geneva was connected to a conductor in Bonn in an effort to rehears “Dèrive” by Pierre Boulez. The system was aimed at “giving the impression to the participants that they are physically in the same room,” and the main challenges were minimizing transmission delay and accurately reproducing the sound space by using multiple microphones and a dummy head. The TransMIDI system (Gang 1997) addresses a similar challenge but instead of sending audio, the system uses the more efficient MIDI protocol that helps minimizing latencies. By using “Transis” group communication system, TransMIDI also allows for easy arrangement of multicast groups so that a “conductor” player can determine exactly what each participant hears at any time. Here too, the system is aimed at bridging the distance between remote participants, allowing them to play, improvise, and listen to music in a similar way to a traditional jam session. The Shaper Approach In the Shaper approach the network’s central system takes a more active musical role by algorithmically generating musical materials and allowing participants to collaboratively modify and shape these materials. Although players in Shaper networks can continuously listen and respond to the music that is modified by all partic ipants, the approach does not support direct algorithmic interdependencies between players. This model can be demonstrated by the Pazellian application (Pazel 2000), a web-based application that uses “Smart Harmony” an algorithmic mechanism that annotates each note with harmonic information and determines a set of harmonic constraints for the composition. Here, players can control parameters such as pitch range, volume, and instrumentation as well as to manipulate multiple individual parameters for all voices in the composition. Players can hear and respond to the musical output that is generated by all the participants, but cannot directly communicate with any specific player. The “Variations for WWW” project (Yamagishi 1998) takes a similar approach. In this system, a Max patch is connected to the web via the W protocol so that remote users can manipulate parameters in an algorithmically generated theme. The Max patch sends MIDI commands to a MIDI synthesizer, which transmits the audio output back to the participants via a Real Networks audio encoder. The system’s interconnectivity is derived from its ability to play the combined manipulation of all users back to the participants, who can modify their musical contribution in response. Here too, the focus is not on generating original material but on modify existing musical content. The Construction Kit Approach this approach offers higher levels of interconnectivity among participants, which are usually skilled musicians, by allowing them to contribute their music to multipleuser composition sessions, manipulate and shape their and other players’ music, and take part in a collective creation. Interaction in such networks is usually asynchronous as participants submit their pre composed tracks and manipulate their peers’ material off-line. Faust Music On Line (Jorda 1999) is a representative example for this approach. Here, a web-based synthesis engine allows players to create musical tracks and construct them into a composition, which then can be downloaded by other participants. If the downloaded composition is not complete (i.e., it still has empty tracks) a participant can generate new tracks locally, add them to the composition, edit them and upload the full piece back to the web. Participants can also reprocess and distort any of the previous tracks in the composition by using a variety of synthesis generators and modifiers. (A commercial paraphrase on this idea is the Rocket Network, (2001). The WebDrum application (Burk 2000) demonstrates a slightly different take on the Construction kit approach by basing the application on a traditional drum pattern editor where users turn on or off notes on a grid. Synthesized drum sounds are used in order to avoid downloading large audio sample files. Web users can play and listen to others participants’ edits and to add their instrument sounds and to their own pallets. The ISX project (Helmuth 2000) combines between the Construction Kit and the Shaper approach by allowing users to algorithmically change their peers’ sounds, as well as to create new tracks from scratch and construct them into a collaborative composition. The project uses Internet2 as wideband platform that can support the exchange of large audio files. The Construction Kit approach provides a high level of interconnectivity by allowing participants to combine their musical materials into compositions and to modify each other’s music. However, the central system in this approach usually plays the role of a static infrastructure as its function is not influenced by the dynamic contributions from participants. Moreover, due to difficulties in controlling Internet latencies , this composition oriented approach cannot really address the live challenges that of fully real-time IMNs. Fig 3. The Construction Kit approach for Internet music collaboration as demonstrated by Sergi Jorda’s “Faust Music On Line.”