8 days touching music: A first experience holding music with hands at RITMO
Two weeks are not enough to explore all the collaboration opportunities that may exist at RITMO, but probably enough to understand that this space is more than suitable to see my research project evolve, make good friends, and dive into a dynamic and fun research environment.
My project started with a first prototype designed at Universidad Indoamérica - Ecuador, and has evolved to an enhanced device optimized at Universidad de Málaga - Spain. The apparatus is capable to convey reliable and robust vibrotactile music compositions. In the first weeks of November, Professor Alexander Refsum Jensenius (Co-Supervisor) gave me the chance to showcase my haptic interface in the robotics cluster, and since then almost 10 researchers at RITMO experienced this way to feel music. Feedback from every single participant in this pilot study, has enriched our knowledge about perception of vibrotactile music and has opened new paradigms and questions we will try to answer next year during a longer research stay.
I consider the comments and recommendations from every single researcher that experienced our approach to vibrotactile music compositions valuable. Just to mention some, I really want to give thanks for the feedback from Dongho Kwak, Stefano Fasciani, Cagri Erdem, Alejandro Omar Blenkmann, Dana Swarbrick, and Fadi Al-Ghawanmeh. I also want to thank for technical support during my research stay from Eirik Slinning Karlsen, Marit Johanne Furunes, Alena Clim, and Kayla Burnim, who made it so easy to get access to the facilities.
But what does music have to do with haptics?
Musical Haptics is an emerging field of research, closely related to haptic feedback in digital musical instruments, although more focused on the hearing-impaired audience. Various approaches for the design of haptic music players (HMPs) have emerged during the last two decades, which can be summarized in three big groups: installations, wearables, and hybrid (find more in our review paper here). Possibly the most challenging fact about conveying music through the skin is the way musical information, such as melody and timbre, is going to be rendered towards vibrotactile feedback. However, I consider more important to answer questions such as: are we communicating the meaning of music? Or, are the emotions evoked by vibrotactile music similar to what is evoked by music for the ears? There are no current straightforward answers to these questions, firstly because no clear method/technology to effectively render music from audio to touch has been developed, and secondly because the affective response from people to vibrotactile music compositions remains unclear.
Still have a lot to do at RITMO
Although the library of vibrotactile music compositions we created this year has positively touched hearing and not hearing participants, the algorithm has not been implemented, so audio to tactile rendering is a challenging and time-consuming task. Next year we will focus on implementing an algorithm as a first try to automatically render music to tactile feedback. Our approach will concentrate on generating meaningful stimuli like what are obtained with music for the ear. We will explore different resources and methods, not only to design the optimal rendering formula but also to validate the affective response on participants. At this stage I am delighted to mention that a master thesis supervised by Professors Jonna Vuoskoski and Alexander Jensenius, and co-supervised by myself has been approved and will contribute to my research project; thanks Alena Clim who will be the person in charge of carrying this exciting work.
Finally, I would like to thank my supervisors from the Universidad de Málaga Professors Fernando Vidal Verdú and Andrés Trujillo León, who have made all the necessary steps to obtain the funds that supported my research stay at University of Oslo, and co-supervisor Professor Alexander Jensenius for giving me the opportunity of being a part of RITMO’s research team.
I’m looking forward to start working at RITMO as a visiting researcher next year, to deeply experiment not only the dynamics of the Research Centre, but the city life, its culture, language, traditions, and all that makes Norway an outstanding place to do research.