The Sound and Music Computing (SMC) discipline aims to design better sound objects and environments for promoting multidisciplinary research to understand, model, and improve human interaction in multimodal domains. Moreover, SMC supports core ICT technologies for the ongoing revolution in digital audio and music culture. In particular, mobile and wireless technologies increasingly promote exciting future developments in SMC. Designing of ubiquitous and distributed interactive spaces defines new concepts and challenges of sound control and reproduction. Mobile and embedded input interfaces allow novel control paradigms. Distributed and wearable sensor systems enable the continuous connection and adaptation between mobile sensing technologies and user data (e.g., physiology, gestures, and location information).
This special issue focuses on interactivity in mobile auditory displays, allowing instantaneous sonic/musical feedback as part of action-perception interaction for users. In particular, the low-latency feedback loop between hardware and software is a key element for facing the complexity of spatio-temporal evolution of sound with relevant implications for mobile interfaces between humans and computers. We devote particular attention to the growing maker communities around open embedded hardware platforms that allow the creation of new communication protocols for audio/multimedia data, musical instruments, and interactive audio systems. In particular, we selected six contributions that cover both technical and theoretical aspects of networked communication for shared sound and music interactions. While two of them deal with the intercommunication problem, the next two publications develop conceptual frameworks for networked music performance in both performative and learning scenarios. The remaining contributions provide interesting insights regarding the interface design process of interactive artifacts and mobile devices considering open platforms.
Roger Dannenberg’s O2: A Network Protocol for Music Systems hits the interoperability problem between music systems by proposing an extension of the popular Open Sound Control protocol. His contribution puts the accent on the problems of interconnection, unreliable message delivery, and clock synchronization; several computer musicians must deal with as part of their routine activities. O2 offers solutions to these problems, furthermore making it straightforward for musicians to migrate their distributed music applications to the new protocol, thanks to sharing its roots with Open Sound Control.
A different intercommunication problem is dealt with Virtual Net: A Decentralized Architecture for Interaction in Mobile Virtual Worlds by B. Shen and J. Guo. In their contribution, the authors put the accent to the scalability problems posed by such mobile virtual worlds as those sharing interactive music content. These worlds in fact must guarantee high interaction responsiveness also in presence of a large number of users.
Their peer-to-peer solution overcomes mobile device unreliability and communication network instability through a novel infrastructure model, called Virtual Net, providing fault-tolerance in user content management and shared object state consistency.
In Interaction Topologies in Mobile-Based Situated Networked Music Systems by B. Matuszewski et al., the authors present a technical framework to support networked music performance (NMP) and systems, as well as theoretical methodological considerations regarding different aspects of interaction (e.g., social and human-computer). Six case studies with mobile devices in different settings from public installations to concerts and performances are then presented to support such a theoretical framework.
S. D Monache et al. work on a different NMP case study regarding learning scenarios within their paper titled A Presence- and Performance-Driven Framework to Investigate Interactive Networked Music Learning Scenarios. The authors detail a conceptual framework for research on a NMP system meant to facilitate shared playing by two musicians in the area of distance and blended learning applications. A preliminary study on chamber music practice meant to explore the effects of latency on presence and quality of the performance in an interactive networked environment.
The Influence of Coauthorship in the Interpretation of Multimodal Interfaces, by F. Morreale et al., addresses the topic of musical interface design from the original perspective of appropriation and even subvertion of interactive systems through multiple coexisting interpretations. The authors introduce a novel design model that can be used to stimulate heterogeneous interpretations of interactive artefacts based on the idea that the design of interpretively flexible systems should embed multiple values and backgrounds at the design stage. The model is illustrated through the case study of Beatfield, a multimodal system, which allows users to control audiovisual material by means of tangible interaction.
Finally, M. Geronazzo et al. propose a portable headphone prototype based on an embedded hardware platform to create an interactive audio story through binaural synthesis. In Creating an Audio Story with Interactive Binaural Rendering in Virtual Reality, the design of two simple interactions based on head-tracking and hand controller aims at demonstrating that the quality of the experience could be highly improved compared to regular static audiobooks. A short story based on the horror narrative of Stephen King’s Strawberry Springs is adapted and designed in virtual environments in order to evaluate the proposed sonic interactivity.
The selected contributions included in this special issue confirm the interest of the SMC research community in taking advantage of the rapid development of hardware and data connectivity for integrating sound and music interactions into mobile and networked devices. Accordingly, it becomes essential to support research into many novel aspects that are crucial for the development of future mobile interactions within the opportunities offered by acoustic data. The upcoming Internet of Things (IoT), together with the 5G network infrastructure, calls for a paradigm shift especially in the social and human-machine interaction with smart objects. Moreover, ubiquitous computing and artificial intelligence algorithms further foster sensory fusion with sound-related environmental information such as event detection, speech communication, collaborative music making, and many more. Finally, augmented reality technologies require communication networks that are able to manage massive amounts of data from both real and virtual worlds simultaneously and always respecting real-time constraints which are crucial for interactions with sound.
Conflicts of Interest
The editors declare that they have no conflicts of interest regarding the publication of this special issue.
Michele Geronazzo
Federico Avanzini
Federico Fontana
Stefania Serafin