The underlying assumption and motivation of sound tracing experiments is that they may reveal salient sound-motion links in perception.
Experiment in 2006
A preliminary study on sound tracing was carried out in 2006, where participants' responses were collected on a digital tablet.
Experiment in 2009
To follow up on the previous experiment, two experiments were carried out in 2009 and 2010, where response data was gathered using optical, infrared marker-based motion capture. Fifteen people participated in the experiment. A rod was equipped with reflective markers, and motion capture data of the rod was collected as response data.
The sound stimuli used in the experiment have been made available for download.
| Sound | Pitch | Spectral Centroid | Loudness | Duration (s) | Onsets |
|---|---|---|---|---|---|
| 1 | Noise | 3 sweeps | 3 sweeps | 3 | 3 |
| 2 | Noise | 3 sweeps | 3 sweeps | 3 | 3 |
| 3 | Falling | Rising | Steady | 2 | 1 |
| 4 | Rising | Falling | Steady | 2 | 1 |
| 5 | Noise | Rising | Steady | 1 | 1 |
| 6 | Noise | Rising / Complex | Steady | 1 | 1 |
| 7 | Noise | Rising, then falling | Steady | 4 | 1 |
| 8 | Rising | Complex | Steady | 2 | 1 |
| 9 | Noise | Steady | Static (on/off) | 2 | 5 |
| 10 | Noise | Complex | Impulsive attacks | 2 | 5 |
Publications:
- Nymoen, Kristian; Caramiaux, Baptiste; Kozak, Mariusz & T?rresen, Jim (2011). Analysing sound tracings: a multimodal approach to music information retrieval, In Cynthia Liem; Meinhard Müller; Douglas Eck & George Tzanetakis (ed.), Proceedings of the 1st international ACM workshop on Music information retrieval with user-centered and multimodal strategies. Association for Computing Machinery (ACM). ISBN 978-1-4503-0986-8. pp 39 - 44
- Nymoen, Kristian; Jensenius, Alexander Refsum; T?rresen, Jim; Glette, Kyrre Harald & Skogstad, St?le Andreas van Dorp (2010). Searching for Cross-Individual Relationships between Sound and Movement Features using an SVM Classi?er , In Kirsty Beilharz; Andrew Johnston; Sam Ferguson & Amy Yi-Chun Chen (ed.), NIME 2010 proceedings: New Interfaces for Musical Expression++, 15-18th June 2010, University of Technology Sydney. Association for Computing Machinery (ACM). ISBN 978-0-646-53482-4. pp 259 - 262
Experiment in 2010
In a new experiment in 2010, an experiment with more participants and more organised sound stimuli was carried out. 38 people participated and traced the sounds using two handles equipped with reflective markers.
Again, more detailed information on the experiment can be found in the publication listed below, and the sound files that were used as stimuli are available for download.
| Sound | Pitch | Spectral Centroid | Loudness | Duration (s) | Onsets |
|---|---|---|---|---|---|
| 1 | Rising | Falling | Bell-shape | 3 | 1 |
| 2 | Falling | Rising | Bell-shape | 3 | 1 |
| 3 | Falling | Falling | Bell-shape | 3 | 1 |
| 4 | Rising | Rising | Bell-shape | 3 | 1 |
| 5 | Rising | Steady | Increasing | 3 | 1 |
| 6 | Falling | Steady | Increasing | 3 | 1 |
| 7 | Steady | Falling | Bell-shape | 3 | 1 |
| 8 | Steady | Rising | Bell-shape | 3 | 1 |
| 9 | Steady | Steady | Increasing | 3 | 1 |
| 10 | Noise | Falling | Bell-shape | 3 | 1 |
| 11 | Noise | Rising | Increasing | 3 | 1 |
| 12 | Noise | Steady | Increasing | 3 | 1 |
| 13 | Steady | Rising slightly | Increasing | 3 | 1 |
| 14 | Steady | Falling slightly | Increasing | 3 | 1 |
| 15 | Rising | Falling | Impulsive | 3 | 1 |
| 16 | Steady | Steady | Impulsive | 3 | 1 |
| 17 | Noise | Steady | Impulsive | 3 | 1 |
| 18 | Noise | Falling | Impulsive | 3 | 1 |
Publications:
- Nymoen, K., Torresen, J., God?y, R. I., and Jensenius, A. R. 2012. A statistical approach to analyzing sound tracings. In S. Ystad et al., Ed. Speech, Sound and Music Processing: Embracing Research in India. LNCS 7172. Berlin Heidelberg: Springer, pp. 120-145