sig-ma.de

The following paper was presented this year at ICASSP 2026 in Barcelona:

Justin Dettmer, Annelot Bosman, Igor Vatolkin, Holger Hoos: Audio Classification Models are Vulnerable to Filter Perturbations 

Abstract: Deep learning models are known to suffer from a lack of robustness to adversarial attacks. While the majority of research on the robustness of neural networks stems from the image classification domain, audio classifiers have also come under scrutiny, as their use in tasks such as speech processing is increasing. We introduce a novel adversarial attack method that creates band-pass filters as perturbations for mel spectrograms. These perturbations are closer to real-life conditions than conventionally created adversarial examples, as they can simulate the use of different recording devices or other acoustic properties. We demonstrate that state-of-the-art audio classifiers lack robustness against these adversarially-created filters, and we show that adversarial training with our attack method can help mitigate this effect on the widely studied NSynth, ESC-50 and SpeechCommands datasets.

Fabian Ostermann, Jonas Kramer, Günter Rudolph: Using Large Language Models as Fitness Functions in Evolutionary Algorithms for Music Generation. Proceedings of the Conference on AI Music Creativity (AIMC), 2025

Abstract: Using evolutionary algorithms for music generation is a well-researched approach. Incorporating language models into that process, however, is new. Recently, creating media from text prompts using machine learning models became a common task. For music, the main research on prompt-based generation addresses direct waveform generation. Works on symbolic music usually use other conditionals. Therefore, our concept of combining evolutionary optimization with the language-music model CLaMP presents the first attempt at prompt-based evolutionary music generation. CLaMP’s ability of estimating similarities between prompts and music is used to design a fitness function. We show that problem-specific mutation and recombination is advantageous compared to classic approaches when it comes to fulfilling desired musical properties. The overall quality of the resulting melodies is currently limited. The reasons are specific blind spots of the CLaMP model. Our concept and code, however, can easily be adapted and used with any other pre-trained model.

M. Gotham, B. Bemman, and I. Vatolkin: Towards an ‘Everything Corpus’: A Framework and Guidelines for the Curation of More Comprehensive Multimodal Music Data. Transactions of the International Society for Music Information Retrieval, 8(1), pp. 70-92.

Abstract: Current state-of-the-art methods for instrument and pitch detection in polyphonic music often require large datasets and long training times; resources which are sparse in the field of music information retrieval, presenting a need for unsupervised alternative methods that do not require such prerequisites. We present a modification to an evolutionary algorithm for polyphonic music approximation through synthesis that uses spectral information to initialise populations with probable pitches. This algorithm can perform joint instrument and pitch detection on polyphonic music pieces without any of the aforementioned constraints. Sets of tuples of (instrument, style, pitch) are graded with a COSH distance fitness function and finally determine the algorithm’s instrument and pitch labels for a given part of a music piece. Further investigation into this fitness function indicates that it tends to create false positives which may conceal the true potential of our modified approach. Regardless of that, our modification still shows significantly faster convergence speed and slightly improved pitch and instrument detection errors over the baseline algorithm on both single onset and full piece experiments.