Tyg: A Single-Screen Environment for “Couch Co-Op” Multiplayer Music-Making

This paper presents Tyg, a single-screen multiplayer audiovisual environment that contains a shared musical instrument system that, after Haraway (1985) and Bidima’s (2012) take on Deleuze, we consider a machine. Implemented in Unreal Engine, this has been designed to enable exploration of and around collaborative real-time creation, multisensory experience, and what we term the “toy-to-instrument” continuum.

Relevant examples of real-time collaborative creation date back as far as The Hub (Gresham-Lancaster, 1998) and Sensorband (Bongers and Sensorband, 1998), and there has been significant subsequent work in the area (Symons, 2023; Vallis et al., 2012; Jordà, 2005a). Drawing on recent ideas in HCI around entanglement (Mudd, 2023; Andersen et al., 2020) and earlier work around player roles (Weinberg, 1998), our focus is on player-player and player-machine interconnections and interdependencies in a shared virtual space. As part of this, we give a brief overview of two initial compositional structures intended to explore group interaction and improvisation dynamics.

While the opacity of much computer-based music performance is well established (Cascone, 2003), video games are almost inherently multisensory (Donohue et al., 2010). We examine design solutions to the problem of readable sonic causality from player and audience perspectives, and outline our approach to multisensory feedback more generally.

The toy-to-instrument continuum relates to a combination of expectations around player engagement, the breadth of sonic possibilities, and the accessibility of (deliberate) sound-making. If sound toys are “quick but shallow”, many traditional musical instruments are “slow but deep” (Jordà, 2005b). We discuss how Tyg challenges this easy continuum by combining aspects usually found at its extremes.

Through discussion of these aspects, the paper seeks to contribute to the ongoing discourse on the intersection of computers and music, multiplayer music systems, and group dynamics in a shared virtual musical environment. It also sets the stage for future research directions.