Abstract
Under-resourced languages (and musics) pose a challenge to machine translation (MT). The challenge is greater when the content of the collected dataset is a varied sample taken from a data population that is even more diverse and dynamic. This is the challenge of Arab music improvisation. We present here the development of a parallel dataset consisting of vocal improvisatory sentences and their corresponding responsive instrumental responses (translations). When developing the dataset, we integrated musicological insights in order to evaluate music theoretical differences between sub-datasets, primarily regarding their size, sentence length, quality, and shared musical identity. We then experimented with MT, comparing several translation models that differ (1) in translation approach (neural versus statistical), and (2) in the dataset handling approach (maqam-based models versus one model for all maqamat). We outline our findings concerning these custom models and subsequently use them as a foundation to consider the future potential for large language models in music translation and background-music generation. Additionally, we delve into how they offer both opportunities and challenges for democratizing and decolonizing music generation.
This work is part of my PhD project performed jointly between the University of Oslo and the University of Lorraine, and supervised by Prof. Alexander Refsum Jensenius and Prof. Kamel Smaili.
Bio
My passion is the design and development of tools for high quality computer-generated (or assisted) music influenced by the maqam tradition. I aim to spread this knowledge via education, performance, and multimedia. I apply Natural Language Processing techniques for AI-based music generation, and also do audio and music signal processing.
I have enjoyed developing Mawaweel, an application that provides automatic accompaniment customized to Arab music. I started with a knowledge-based model when completing my MA in Music Technology at New York University. I continued my research when I joined the University of Jordan’s Music Department as an instructor for eight years. In the past period, I started investigating music generation using Machine Translation (MT). This led me to my current PhD research at the University of Oslo and the University of Lorraine, where I am working on MT-based music generation with individual customization.