When
Thematic Session 5: Mapping and Control (Tuesday, 15:30)
Abstract
DanzArTe is a treatment protocol and an interactive system designed mainly for older people at risk of fragility. It is grounded on the active experience and real-time processing of visual arts content (baroque religious paintings, image manipulation and 3D modelling), and interactive sonification, to support full-body physical activity and cognitive exercise of memory training. We propose to present the interactive sonification model of full-body (individual as well as joint) human expressive movement used in this project. Our model is grounded on our Slow Mood and Aesthetic Resonance paradigms. Movement data include Quantity of Motion, Smoothness, and Synchronization of full-body users' movements, from real-time video analysis in EyesWeb, using a low-cost sensor system (Kinect II). The model of interactive sonification of human movement qualities is based on cross-modal mappings of movement qualities at different levels of abstraction and temporal scales. This multi-layer multi-time model covers from slowly evolving interactive environmental soundscapes (from several seconds to tenth of seconds) to mid-level expressive features in ranges of a few seconds and of half a second, to real-time mappings. The main interaction design guidelines can be summarized as follows: (i) Multiple-layer cross-modal correspondence between movement and sound; (ii) Dynamically evolving environmental soundscapes following slowly evolving movement features; (iii) Slowness and continuity in dynamic curves; (iv) Low-intrusiveness, semi-conscious interactive sonification; (v) Subtractive design; (vi) Use of Silence; (vii) Polyphony and orchestration techniques to the sonification of joint actions in dyads and small groups. Our approach aims to offer a novel paradigm for interactive sonification, intended as a way to contribute to a deep experience of the expressive gesture of characters in visual art paintings, to entrain and express similar qualities in full-body joint movements: the body as a means to induce a deeper experience of the painting.
Bio
Myself (Andrea Cera) and professor Antonio Camurri (head of Infomus - CasaPaganini) plan to attend online for a joint presentation. We would like to present also a couple of short videos where it is possible to see the DanzArTe system at work.