Fibres Out of Line is an interactive art installation and performance for the 2021 Rhythm Perception and Production Workshop (RPPW). Visitors can watch the performance and subsequently interact with the installation, all remotely via Zoom.
Setup
The Fibres Out of Line installation consists of 10 autonomous musical agents, set up as shown in Figure 1.
There are eight Dr. Squiggles musical robots that play acoustic sound and two software agents that play sound through speakers. These break down as follows:
- Dr. Squiggles
- 2 Glockenspiel Robots
- 3 Tapping Robots
- 3 Pipe Organ Robots
- Software Agents
- CAVI
- RAVI
More general information about how these agents are constructed and behave can be found in the following videos and links. (a) (b) Figure 2: Videos depicting some of the individual agents used in the setup, showing (a) Dr. Squiggles Glockenspiel robots and (b) Dr. Squiggles Organ Pipe robots.
We wrote a Make Magazine tutorial so you can build your own Dr. Squiggles. Learn more about the Dr. Squiggles Project.
Windchime Mode
The installation has two interactive modes. By default, Windchime Mode is active. A functional diagram of this mode is in Figure 3.
Windchime Mode behaves as follows:
- Two glockenspiel robots watch the Zoom video stream. They divide the video into eight columns, and if they see sufficient movement in any of the columns, they will play a note corresponding to that particular column.
- Three pipe organ robots listen to the environment, each with its own omnidirectional microphone. Each pipe organ robot tries to play whatever pitch it hears most prominently.
- CAVI listens to the glockenspiel robots and plays a transformed version of what it hears. It uses time-warping, pitch-shifting, delay, reverb, and other effects to produce the transformed audio.
Rhythmic Mode
Rhythmic Mode is activated when the performer or visitor waves their hand around rapidly in front of their camera for a few seconds. It is deactivated when there is no movement at all in the camera for a few seconds. A functional diagram of Rhythmic Mode is shown in Figure 4.
Rhythmic Mode behaves as follows:
- RAVI generates random rhythms using tabla-like sounds. The tempo and note density depend on how much movement RAVI sees in the Zoom video stream. Moreover, sudden movement will cause RAVI to play a battery of metallic banging and crashing sounds.
- Two glockenspiel robots play a variant of what RAVI plays. These robots do not behave autonomously in this mode; instead, these two robots, together with RAVI, behave as a single agent governed by one computer.
- Three tapping robots listen to the environment, each via its own omnidirectional microphone. Each robot independently tries to synchronize to the beat and tap out similar rhythms but not identical to what it hears. These robots also blink to the beat.
- Three pipe organ robots behave the same as the tapping robots, except they make sounds by playing short bursts of air through the organ pipes instead of tapping on things.
- CAVI is awesome.
More information about how the tapping robots track the beat and generate rhythms is published in the Journal of New Music Research: Read the Article Online.
Streaming
The entire setup is streamed over Zoom via several cameras and microphones, as illustrated in Figure 5. This allows the performer and participants to interact with the installation through Zoom video and simultaneously view and hear the installation. Many independent devices with cameras are also set up, which will join the Zoom session as participants, and participants can view different scenes (close and wide shots) of the installation. The performance will be recorded at several locations: Zoom, local PC (OBS), Tablet camera (SD card), and Garmin VIRB 360 (SD card).
Performance
On 22 June at 21:40 CET on Zoom, the performer Jen Gerry will improvise a ?10 minute dance performance with the robots. Jen is in California, while the robots are in Oslo. Both Jen and the Robots will join a Zoom session, so they can see, hear, and respond to one another as described above. This is depicted in Figure 6. Figure 6: The performance of Fibres Out of Line. You can read more about Jen on her bio page: Jen's Bio RPPW participants can join via Zoom to behold the performance.
Installation
Immediately following the performance and continuing for the remainder of RPPW, participants will have the opportunity to join a Zoom room and interact with the robots. This is depicted in Figure 7.
The system works best with one participant at a time, so it might be chaotic immediately after the performance as many people may attempt to interact simultaneously. Participants may have a better experience if they come back later at a random time. RPPW attendees will be emailed the Zoom link.
Enable Stereo Audio
The performance and installation will sound better if you enable stereo audio in Zoom. If you don't do this, you may only hear the mono sound. There is an option for this in the audio settings, as shown in Figure 8.
Credits
The installation and performance are the result of a collaboration between the RITMO Center for Interdisciplinary Studies in Rhythm, Time, and Motion at the University of Oslo: RITMO and the Driven Arts Collective: DAC.
The performance was hosted by the Shawl-Anderson Dance Center: SADC and was shown at RPPW 2021: RPPW.
No contact mics were harmed in the making of this piece.