Lecture @ RWTH Aachen by Nikita Braguinski: Life-Like Artificial Music: Understanding the Impact of AI on Musical Thinking

Life-Like Artificial Music: Understanding the Impact of AI on Musical Thinking
June 5, 17:00-18:30
RWTH Aachen. Käte Hamburger Kolleg “Cultures of Research”
Theaterstraße 75, 52062 Aachen
Online participation is also possible: https://khk.rwth-aachen.de/event/evening-lecture-ss24-4/

This lecture explores the impact of machine learning on the future of music research and theory.  It argues that AI-generated music poses a deep challenge for existing theories: AI systems can  learn to imitate musical styles without receiving any information about human music theory concepts, raising questions about the validity of those concepts. Additionally, music-generating  AI systems can be trained on audio directly, bypassing notation, while human music theory  almost always works with notation as a simplified and abstracted proxy.

As an example of the conceptual challenges and shifts that now arise in music research, the talk  examines a recent paper that compares Western music theory concepts with structures that  emerge in a machine learning model trained on musical notation. While the paper finds  similarities between the two, the talk argues that the machine learning system’s output is still  influenced by human biases and choices in the training data and model architecture – and that  this influence may in fact be unavoidable.

Finally, the talk argues that while AI may be able to generate novel structures for analyzing
music, their applicability to human music theory and practice may prove to be extremely limited  due to the differences between human cognition and machine learning. Overall, the talk raises  questions about the future potential for AI to disrupt human theory-making – and not only in the  discipline of musicology.

No knowledge of musicological concepts is required for understanding the presentation and participating in the discussion.

Dr. Nikita Braguinski is a 2023-2024 Fellow at the Käte Hamburger Kolleg “Cultures of Research” at RWTH Aachen University. In his work he currently concentrates on the possible impact of machine learning and big online listening datasets on the future of music research. His book “Mathematical Music. From Antiquity to Music AI” (Routledge, 2022) was translated into Korean, receiving the Sejong book prize in 2023. He was a Fellow at Harvard University, a Visiting Scholar at the University of Cambridge, and a Researcher at Humboldt University of Berlin with funding from the Volkswagen Foundation. In 2023, he co-convened, together with Eamonn Bell and Miriam Akkermann, the ZiF Bielefeld Visiting Research Group “The Future of Musical Knowledge in the Age of Machine Learning”.

Veröffentlicht unter Events | Hinterlasse einen Kommentar

Call for Papers: TISMIR Special Collection on Multi-Modal Music Information Retrieval

TISMIR Special Collection on Multi-Modal Music Information Retrieval
(see also the PDF version on the TISMIR web page)

Deadline for Submissions
01.08.2024

Scope of the Special Collection
Data related to and associated with music can be retrieved from a variety of sources or modalities: audio tracks; digital scores; lyrics; video clips and concert recordings; artist photos and album covers; expert annotations and reviews; listener social tags from the Internet; and so on. Essentially, the ways humans deal with music are very diverse: we listen to it, read reviews, ask friends for recommendations, enjoy visual performances during concerts, dance and perform rituals, play musical instruments, or rearrange scores.

As such, it is hardly surprising that we have discovered multi-modal data to be so effective in a range of technical tasks that model human experience and expertise. Former studies have already confirmed that music classification scenarios may significantly benefit when several modalities are taken into account. Other works focused on cross-modal analysis, e.g., generating a missing modality from existing ones or aligning the information between different modalities.

The current upswing of disruptive artificial intelligence technologies, deep learning, and big data analytics is quickly changing the world we are living in, and inevitably impacts MIR research as well. Facilitating the ability to learn from very diverse data sources by means of these powerful approaches may not only bring the solutions to related applications to new levels of quality, robustness, and efficiency, but will also help to demonstrate and enhance the breadth and interconnected nature of music science research and the understanding of relationships between different kinds of musical
data.

In this special collection, we invite papers on multi-modal systems in all their diversity. We particularly encourage under-explored repertoire, new connections between fields, and novel research areas. Contributions consisting of pure algorithmic improvements, empirical studies, theoretical discussions, surveys, guidelines for future research, and introductions of new data sets are all welcome, as the special collection will not only address multi-modal MIR, but also cover multi-perspective ideas, developments, and opinions from diverse scientific communities.

Sample Possible Topics
● State-of-the-art music classification or regression systems which are based on several
modalities
● Deeper analysis of correlation between distinct modalities and features derived from them
● Presentation of new multi-modal data sets, including the possibility of formal analysis and theoretical discussion of practices for constructing better data sets in future
● Cross-modal analysis, e.g., with the goal of predicting a modality from another one
● Creative and generative AI systems which produce multiple modalities
● Explicit analysis of individual drawbacks and advantages of modalities for specific MIR tasks
● Approaches for training set selection and augmentation techniques for multi-modal classifier systems
● Applying transfer learning, large language models, and neural architecture search to
multi-modal contexts
● Multi-modal perception, cognition, or neuroscience research
● Multi-objective evaluation of multi-modal MIR systems, e.g., not only focusing on the quality, but also on robustness, interpretability, or reduction of the environmental impact during the training of deep neural networks

Guest Editors
● Igor Vatolkin (lead) – Akademischer Rat (Assistant Professor) at the Department of Computer Science, RWTH Aachen University, Germany
● Mark Gotham – Assistant professor at the Department of Computer Science, Durham
University, UK
● Xiao Hu – Associated professor at the University of Hong Kong
● Cory McKay – Professor of music and humanities at Marianopolis College, Canada
● Rui Pedro Paiva – Professor at the Department of Informatics Engineering of the University of Coimbra, Portugal

Submission Guidelines
Please, submit through https://transactions.ismir.net, and note in your cover letter that your paper is intended to be part of this Special Collection on Multi-Modal MIR.
Submissions should adhere to formatting guidelines of the TISMIR journal:
https://transactions.ismir.net/about/submissions/. Specifically, articles must not be longer than 8,000 words in length, including referencing, citation and notes.

Please also note that if the paper extends or combines the authors‘ previously published research, it is expected that there is a significant novel contribution in the submission (as a rule of thumb, we would expect at least 50% of the underlying work – the ideas, concepts, methods, results, analysis and discussion – to be new).

In case you are considering submitting to this special issue, it would greatly help our planning if you let us know by replying to igor.vatolkin [AT] rwth-aachen.de.

Veröffentlicht unter Conferences & Calls | Hinterlasse einen Kommentar

SIGMA #56

The program of the following 56th SIGMA meeting on 15.02.2024, 14:00-16:20, which takes place at the Chair for AI Methodology, RWTH Aachen, Theaterstr. 35-39, room 325, and online (please send an email to igor.vatolkin [at] rwth-aachen.de if you wish to get the Zoom link):

14:00-14:05 Welcome greetings

14:05-14:30 Conference study
Fabian Ostermann: Adaptive video game music as a multi-objective benchmark for conditional autoregressive models

14:30-15:20 Research study
Claus Weihs: Optimized decision trees – how to improve model quality in music data analysis

15:20-16:10 Research discussion
Martin Ebeling: Is that what you hear? How ambiguities in hearing disturb the modelling of auditory perception

16:10-16:20 Conferences and calls, teaching activities, miscellaneous, next meeting

Veröffentlicht unter SIGMA | Hinterlasse einen Kommentar

Two papers accepted for EvoMUSART 2024

(1) J. Dettmer, I. Vatolkin, and T. Glasmachers: Weighted Initialisation of Evolutionary Instrument and Pitch Detection in Polyphonic Music. Accepted for Proceedings of the 13th International Conference on Artificial Intelligence in Music, Sound, Art and Design (EvoMUSART)

Abstract: Current state-of-the-art methods for instrument and pitch detection in polyphonic music often require large datasets and long training times; resources which are sparse in the field of music information retrieval, presenting a need for unsupervised alternative methods that do not require such prerequisites. We present a modification to an evolutionary algorithm for polyphonic music approximation through synthesis that uses spectral information to initialise populations with probable pitches. This algorithm can perform joint instrument and pitch detection on polyphonic music pieces without any of the aforementioned constraints. Sets of tuples of (instrument, style, pitch) are graded with a COSH distance fitness function and finally determine the algorithm’s instrument and pitch labels for a given part of a music piece. Further investigation into this fitness function indicates that it tends to create false positives which may conceal the true potential of our modified approach. Regardless of that, our modification still shows significantly faster convergence speed and slightly improved pitch and instrument detection errors over the baseline algorithm on both single onset and full piece experiments.

(2) L. Fricke, M. Gotham, F. Ostermann, and I. Vatolkin: Adaptation and Optimization of AugmentedNet for Roman Numeral Analysis Applied to Audio Signals. Accepted for Proceedings of the 13th International Conference on Artificial Intelligence in Music, Sound, Art and Design (EvoMUSART)

Abstract: Automatic music harmony analysis has recently been significantly improved by AugmentedNet, a convolutional recurrent neural network for predicting Roman numeral labels. The original network receives perfect note annotations from the digital score as inputs and predicts various tonal descriptors: key, chord root, bass note, harmonic rhythm, etc. However, for many music tracks the score is not available at hand. For this study, we have first adjusted AugmentedNet for a direct application to audio signals represented either by chromagrams or semitone spectra. Second, we have implemented and compared further modifications to the network architecture: a preprocessing block designed to learn pitch spellings, increase of the network size, and addition of dropout layers. The statistical analysis helped to identify the best among all proposed configurations and has shown that some of the optimization steps significantly increased the classification performance. Besides, AugmentedNet can reach similar accuracies with audio features as inputs, compared to the perfect annotations that it was originally designed for.

Veröffentlicht unter General | Hinterlasse einen Kommentar

Proseminar on AI in music data analysis

During the summer term 2024, a proseminar „Artificial Intelligence in Music Data Analysis“ will take place the Chair for Artificial Intelligence Methodology, RWTH Aachen.

Veröffentlicht unter Teaching Activities | Hinterlasse einen Kommentar

Job offer as student assistant at the Chair for AI Methodology, RWTH

A position as student assistant for the support in teaching activities and empirical studies for optimisation of music data analysis using Artificial Intelligence is open and should be filled as soon as possible. For details, please see the PDF.

Veröffentlicht unter Jobs | Hinterlasse einen Kommentar

Call for Papers: EvoMUSART 2024

The 13th International Conference on Artificial Intelligence in Music, Sound, Art and Design (EvoMUSART) will take place on 3-5 April 2024, as part of the evo* event.

EvoMUSART webpage: www.evostar.org/2024/evomusart/

Submission deadline: 1 November 2023

Conference: 3-5 April 2024

EvoMUSART is a multidisciplinary conference that brings together researchers who are working on the application of Artificial Neural Networks, Evolutionary Computation, Swarm Intelligence, Cellular Automata, Alife, and other Artificial Intelligence techniques in creative and artistic fields such as Visual Art, Music, Architecture, Video, Digital Games, Poetry, or Design. This conference gives researchers in the field the opportunity to promote, present and discuss ongoing work in the area.

Submissions must be at most 14 pages long, excluding references, in Springer LNCS format. Each submission must be anonymized for a double-blind review process.

Accepted papers will be presented orally or as posters at the event and included in the EvoMUSART proceedings published by Springer Nature in a dedicated volume of the Lecture Notes in Computer Science series.

More information on the submission process and the topics of EvoMUSART: www.evostar.org/2024/evomusart/

Flyer of EvoMUSART 2024:
http://www.evostar.org/2024/flyers/evomusart

Veröffentlicht unter Conferences & Calls | Hinterlasse einen Kommentar

SIGMA #55

The program of the following 55th SIGMA meeting on 16.06.2023, 14:00-16:00, which takes place at the Chair of Algorithm Engineering, Deparment of Computer Science, TU Dortmund, Otto-Hahn-Str. 14, room 202, and online (please send an email to igor.vatolkin [at] udo.edu if you wish to get the Zoom link):

14:00-14:05 Welcome greetings

14:05-14:35 Bachelor’s thesis (results)
Alhuseen Ali: Comparison between Artificial Neural Networks and Traditional Classifiers for the Suppression of Background Noise

14:35-15:05 Research study
Leonard FrickeApplication of AugmentedNet for Different Sound Bodies

15:05-15:35 Conference study
Igor Vatolkin: Musical Genre Recognition based on Deep Neural Network Predictions of Harmony, Instrumentation, and Segments

15:35-15:55 Conferences and calls, miscellaneous, next meeting

Veröffentlicht unter SIGMA | Hinterlasse einen Kommentar

Paper describing Artificial Audio Multitracks (AAM) dataset published in EURASIP Journal on Audio, Speech, and Music Processing

F. Ostermann, I. Vatolkin, and M. Ebeling: AAM: a Dataset of Artificial Audio Multitracks for Diverse Music Information Retrieval Tasks. EURASIP Journal on Audio, Speech, and Music Processing, 13, 2023.

Zenodo link: https://doi.org/10.5281/zenodo.5794629

Abstract: We present a new dataset of 3000 artificial music tracks with rich annotations based on real instrument samples and generated by algorithmic composition with respect to music theory. Our collection provides ground truth onset information and has several advantages compared to many available datasets. It can be used to compare and optimize algorithms for various music information retrieval tasks like music segmentation, instrument recognition, source separation, onset detection, key and chord recognition, or tempo estimation. As the audio is perfectly aligned to original MIDIs, all annotations (onsets, pitches, instruments, keys, tempos, chords, beats, and segment boundaries) are absolutely precise. Because of that, specific scenarios can be addressed, for instance, detection of segment boundaries with instrument and key change only, or onset detection only in tracks with drums and slow tempo. This allows for the exhaustive evaluation and identification of individual weak points of algorithms. In contrast to datasets with commercial music, all audio tracks are freely available, allowing for extraction of own audio features. All music pieces are stored as single instrument audio tracks and a mix track, so that different augmentations and DSP effects can be applied to extend training sets and create individual mixes, e.g., for deep neural networks. In three case studies, we show how different algorithms and neural network models can be analyzed and compared for music segmentation, instrument recognition, and onset detection. In future, the dataset can be easily extended under consideration of specific demands to the composition process.

Veröffentlicht unter MIR Research, Publications | Hinterlasse einen Kommentar

Slides from the MDA lecture by Meinard Müller

On 03.02.2023, the last unit of  the interdisciplinary lecture on Music Data Analysis was organized by Prof. Meinard Müller, Audiolabs Erlangen. The slides on FMP Notebooks are available here (handout version).

Veröffentlicht unter Teaching Activities | Hinterlasse einen Kommentar