Special Issue on Digital Audio Effects

1. Introduction

Digital audio effects (DAFx) play a constantly increasing role in music, which inspires their design and is branded in its turn by their peculiar action [1]. In this sense, they represent a unique touchpoint, where creation and engineering meet within the rules and technologies of digital sound. From musical signal analysis and synthesis until music production and from the traditional world of acoustics to groundbreaking research in machine listening, innovations in such effects are increasingly specialised and advanced, and unite music technicians, sound specialists, and applied psychologists under the broader DAFx research field.

This Special Issue publishes both original papers and review articles on the diverse topics forming this field. Seven of these submissions expanded and improved previously accepted contributions to the 22nd International Conference on Digital Audio Effects (DAFx 2019, 2–6 September 2019, Birmingham, UK). The remaining five papers include three new scientific articles, which have not been presented in any conference previously, a historical survey of audio effects, especially welcome in this Special Issue, and a meeting report of the DAFx 2019 conference. The next sections gather these scientific publications under topics that had been called at the DAFx 2019, providing a short interpretation of their content within the respective topic. Together with the meeting report [2], all these paper can be accessed at https://www.mdpi.com/journal/applsci/special_issues/Digital_Audio.

2. Review Article

A separate place is reserved here for A History of Audio Effects, a review article by Thomas Wilmering, David Moffat, Alessia Milo, and Mark B. Sandler [3], which discusses the development and evolution of audio effect technology along with highlighting major technical breakthroughs and the impact of available audio effects. This article describes also many analog techniques predating current digital methods.

3. Capture and Analysis of Audio and Music

Two papers included in this Special Issue deal with automatic content analysis of music, in this case the retrieval of music and the recognition of emotions. These papers have not been included in DAFx conferences. In spite of their different scopes, they share methodological aspects, the use of principal component methods and neural networks to perform the analysis.

Specifically, Learning Low-Dimensional Embeddings of Audio Shingles for Cross-Version Retrieval of Classical Music by Frank Zalkow and Meinard Müller [4] shows through robust experiments that, when using neural networks, one can strongly reduce the audio shingle dimensionality with only a moderate loss in retrieval accuracy compared to the use of traditional principal component analysis. In Musical Emotion Recognition with Spectral Feature Extraction Based on a Sinusoidal Model with Model-Based and Deep-Learning Approaches [5], Baijun Xie, Jonathan C. Kim, and Chung Hyuk predict the level of arousal and valence in music from specific spectral features using traditional regression models and, then, deep convolutional neural networks.

4. Effects and Manipulation of Musical Sound

Traditionally one of the most crowded topics at DAFx conferences, the effects and manipulation of musical sound have recently welcomed the application of artificial intelligence to digital sound processing. In addition, three out of four papers in this category loom into the machine learning realm.

Deep Learning for Black-Box Modeling of Audio Effects by Marco A. Martínez Ramírez, Emmanouil Benetos and Joshua D. Reiss [6] testifies the disruption of the traditional analog model-based approach to DAFx design, in favor of black-box strategies based on neural networks. The authors analyse different state-of-the-art deep learning models and explore their objective and subjective performance in various analog audio effect case studies.

The use of specific deep neural networks is detailed in Real-Time Guitar Amplifier Emulation with Deep Learning by Alec Wright, Eero-Pekka Damskägg, Lauri Juvela, and Vesa Välimäki [7]. This work challenges black-box modelling of guitar amplifiers and distortion pedals, coming to convincing real-time emulations of such highly nonlinear audio distortion circuits, with little or no differences perceived by listeners compared to the original.

Third-Octave and Bark Graphic-Equalizer Design with Symmetric Band Filters by Jussi Rämö, Juho Liski, and Vesa Välimäki [8] uses an iterative weighted least-squares method to optimize the filter gains in a graphic equalizer, outperforming previous state-of-the-art, and also applies a recently proposed neural gain control relying on a two-level perceptron ensuring much faster optimization and only a slightly worse accuracy than the least squares method.

Sebastian J. Schlecht proposes a traditional yet innovative derivation in his paper entitled Frequency-Dependent Schroeder Allpass Filters [9]. The author extends the renowned Schroeder allpass filter by introducing frequency-dependent feedforward and feedback gains, which maintain the allpass property. For its generality and computational efficiency, this extension improves upon previous designs as exemplified by a decorrelation filter with frequency-dependent decay characteristics.

5. Hardware and Software Design for Digital Audio Effects

Hardware and software realizations occasionally appear at the DAFx conference. In Multisensory Plucked Instrument Modeling in Unity3D: From Keytar to Accurate String Prototyping by Federico Fontana, Razvan Paisa, Roberto Ranon, and Stefania Serafin [10], Keytar is presented as a plucked guitar simulation mockup developed with the Unity3D environment, providing auditory, visual, and haptic feedback to the player through a Phantom Omni robotic arm, with the goal of designing, prototyping, and testing multisensory interactions resulting from plucked instrument models.

6. Representation, Transformation, and Modelling of Audio Signals

Anti-aliasing methods are sometimes brought to the foreground, as part of audio signal transformations, to increase the accuracy in nonlinear signal processing. The winner of the best paper award at the DAFx 2019 conference, Antiderivative Antialiasing for Stateful Systems by Martin Holters [11], extends the recent anti-derivative method to systems with memory, which are not limited to single one-port nonlinearities. The proposed method is applied successfully to the modeling of two guitar distortion effect circuits, which are popular subjects in the DAFx community.

7. Sound Synthesis, Composition, and Sonification

Similarly to effects processing, sound synthesis has been a core subject at DAFx conferences from the beginning. Flow Synthesizer: Universal Audio Synthesizer Control with Normalizing Flows by Philippe Esling, Naotake Masuda, Adrien Bardet, Romeo Despres, and Axel Chemla-Romeu-Santos [12] highlights the importance of affordable, intuitive control in today’s complex sound synthesizers. By using Variational Auto-Encoders and Normalizing Flows, the authors invert the synthesizer’s map from its controls to a “latent” audio space, this way inferring parameters, learning control, drawing smooth transitions, and defining presets. The automatic design of such inverse maps is implemented in real time on an open-source Max4Live device.

8. Spatial Sound Analysis, Coding, and Synthesis

Not previously communicated to the DAFx 2019 either, the paper entitled Denoising Directional Room Impulse Responses with Spatially Anisotropic Late Reverberation Tails, by Pierre Massé, Thibaut Carpentier, Olivier Warusfel, and Markus Noisternig [13], shows that an optimized spatial domain obtained by plane-wave decomposition can be de-noised so as to preserve the peculiar aural characteristics of highly anisotropic yet incoherent late reverberation fields.

9. Conclusions

The richness and breadth of the submitted material to this Special Issue testifies the liveliness of the research on digital audio effects, and their continuing importance in the musical creation process. It can be anticipated that more novel ideas are yet to appear in the next DAFx conference editions, hopefully worth being attended also by researchers who normally focus their effort outside the music field. The design ideas and methods animating the research in digital audio processing remain highly exciting. We hope that this collection will serve as an inspiration for more research in digital audio effects in the future.