Fabian Ostermann, Jonas Kramer, Günter Rudolph: Using Large Language Models as Fitness Functions in Evolutionary Algorithms for Music Generation. Proceedings of the Conference on AI Music Creativity (AIMC), 2025
Abstract: Using evolutionary algorithms for music generation is a well-researched approach. Incorporating language models into that process, however, is new. Recently, creating media from text prompts using machine learning models became a common task. For music, the main research on prompt-based generation addresses direct waveform generation. Works on symbolic music usually use other conditionals. Therefore, our concept of combining evolutionary optimization with the language-music model CLaMP presents the first attempt at prompt-based evolutionary music generation. CLaMP’s ability of estimating similarities between prompts and music is used to design a fitness function. We show that problem-specific mutation and recombination is advantageous compared to classic approaches when it comes to fulfilling desired musical properties. The overall quality of the resulting melodies is currently limited. The reasons are specific blind spots of the CLaMP model. Our concept and code, however, can easily be adapted and used with any other pre-trained model.