Deep Learning Reshapes Audio Effects: Neural Network Black-Box Modeling of Multiband Saturators

This project was published by joao-canais on GitHub (link: https://github.com/joao-canais/Black-Box-Modelling-of-Multiband-Saturation). Its core is to use deep neural networks for black-box modeling of the FabFilter Saturn 2 multiband saturator, comparing the performance of two architectures—bidirectional LSTM and WaveNet-style dilated causal convolution—in electric bass audio processing. The project uses the IDMT-SMT-Bass dataset, optimizes via multi-dimensional loss functions, and provides online audio demos to verify the results.

Background

Virtual analog modeling is a popular direction in the audio processing field. Digital music production relies on software effects, but classic hardware is expensive and hard to obtain. Multiband saturators are extremely difficult to model due to their frequency-dependent nonlinear characteristics.

Motivation

Focus on black-box modeling of the FabFilter Saturn 2 plugin (without analyzing internal structures, only learning transformations through input and output). This method can be applied to any closed-source commercial plugin and has strong versatility.

Dataset

Uses the IDMT-SMT-Bass dataset from Fraunhofer IDMT, which contains about 5200 electric bass WAV files. Electric bass was chosen because of its rich spectrum (low-frequency fundamental tones + high-frequency overtones), which can fully test the effect of multiband processing.

Experimental Design

Follows the supervised learning paradigm: Process original audio with specific Saturn 2 presets to generate clean/saturated paired samples as training data, allowing the model to learn input-output mapping.

Bidirectional LSTM

A classic recurrent neural network variant with a gating mechanism to capture long-term sequence dependencies. The bidirectional design considers both past and future contexts, making it suitable for audio time-series signals.

WaveNet-style Dilated Causal Convolution

An architecture proposed by DeepMind. Through dilated convolution, it captures extremely long-distance dependencies while maintaining causality (using only past information), and has been proven to generate high-quality natural audio.

Loss Functions

Uses auraloss combined loss:

1. ESR (Error Signal Ratio): Measures time-domain waveform reconstruction accuracy;
2. DC Loss: Prevents DC offset;
3. MRSTFT (Multi-Resolution STFT Loss): Evaluates spectral features from the frequency domain, aligning with human auditory perception.

Experimental Results

Provides online audio demos to compare original effects with model outputs, demonstrating a mature paradigm for end-to-end waveform modeling (dataset selection → architecture design → multi-dimensional loss → verifiable demos).

Technical Insights

The black-box modeling approach can be transferred to various audio devices such as guitar amplifiers, reverbs, and compressors. Developers can quickly prototype effects, and users can obtain high-end sound quality at low cost.

Future Outlook

With the improvement of inference efficiency and the development of model compression technology, such deep learning effects are expected to move from prototypes to products, driving technological innovation in the music production field.