Spatial Sound Hacklab at ZKM GLOBALE:
A Review (2016)
Paul Oomen
As part of ZKM GLOBALE, 4DSOUND hosted its second edition of the Spatial Sound Hacklab in cooperation with TU Berlin and UdK Berlin. Over the course of four days, twenty selected coders, creators and performers explored new instrumental approaches, performance tools and conceptual frameworks in working with spatial sound.
The Spatial Sound Hacklab is 4DSOUND’s essential meet-up with the community. While participants are getting to know the system and each other in a very short timeframe, things culminate with dazzling speed - a spontaneous take on our software and controls, inspiration and ideas for new tools and approaches and highly energetic discussion about the field are instantly emerging.
During our first edition (covered in this video feature), hosted in collaboration with CDM and Fiber at ADE Sound Academy in October 2014, we were able to walk through the sonified history of the stock exchange, discover a solar system with our ears and play a game of sonic pong in the space, among many other luminous ideas.
Here’s a roundup of what happened during the second edition.
4HANDS FOR 4DSOUND
Composers and live-electronic performers Mathieu Chamagne and Hervé Birolini from Nancy, France, teamed up for an impressive performance. 4Hands for 4DSound was performed using LeapMotion and a Wii-mote with patches running from their own systems built in Max/MSP and audio engine Usine that communicated directly with 4DSOUND’s spatial engine.
Said Chamagne: “it was very easy to adapt my patches to control 4DSOUND with OSC messages, and when using the LeapMotion I really had the feeling of playing the space with my fingers.”
Interesting in this approach is the tight link between gestural control and the spatialization of the sound design. Different drums, strings and noises were given a place in a virtual space and the movement of the hands and fingers then performs those sounds dynamically. Integrating the 4DSOUND control in their code enabled them to place those virtual sound objects physically in different places, trigger the sounds from all corners and sides, grab the sound and literally throw the sounds through the space.
KARLSRUHE METRIC
Emiddio Vasquez, a mathematician and musician originally from Cyprus, based in London, explored his interest in topology and sound in his piece ‘Karlsruhe Metric’. He used 4DSOUND’s setup of Max4Live devices in Ableton to construct a symmetric system of concentric rings according to the measures of the city plan of Karlsruhe.
During a 10-minute live performance the centered and strictly symmetrical sound structure became decentralized and fell apart into multiple shreds, bits and pieces floating through the air. Notions of the history and geography of the actual environment in where the event took place merged nicely with the abstract notions of mathematical spatial relationships as part of this conceptual sound experience.
TEAM AAK'D
Team AAK’D (‘acked) decided to focus on narrative and what new forms of musical storytelling can be explored using sound in space only, eliminating the visual sense by performing in pitch dark. The team consisted of electronic musicians Daniel Neves and André Hencleeday from Lisbon, installation and media artist Katherine Bennett from New York and Austin Larkin, a mixing engineer based in Berlin developing himself as a conceptual artist.
I compose music like sculpting a sculpture, I shape the sounds into something, and the system gave me a spatial approach to that kind of composition. Beside that, I also believe that due to its structure which allows the audience to be inside the system, it gives more value to the sound and to the music, which is something I strongly defend. When you are inside the system you are there, focused on what is happening, and I wish that this can soon be the future of music, systems that value the music, systems that allow sound to be only sound and nothing more.
In general I’m not a great fan of visuals in music performances, because they distract us from the sound, and systems like 4DSOUND allows the sound to act alone.
— Daniel Neves
The entire team performed as a group playing spatial sound designs from within Ableton and live performance using 4DSOUND’s template in Lemur for control on the iPad and the custom hardware controller Space Control 01.
AIDKID
Often, working with spatial sound means the artist starts from scratch and feels urged to reinvent his own methods to making music. A different take on using spatial sound can be to explore the role of spatial sound in more established approaches to sound productions. How can these approaches be enriched by it? And what are the spatial forms that belong to those elements regularly found in compositions, like a bass drum, percussion, melody and harmonic pads?
Ondřej Mikula from Czech republic, a.k.a. AidKid, was exploring just that when he mixed one of his richly layered tracks with bell sounds, piano, percussion and beats in 4DSOUND. His mix, realized using the 4D.Animator software shows the tendency of a kick-drum to be static and with wide dimensions in the middle, while the mid-to-higher frequency components of the composition become increasingly dynamic and moving through and around the space. Then, by adding different types of reflecting walls that produce a lively spatial reverb, he lifted the composition to a more ephemeral place.
NOISE TEAM - SPATIAL GLITSCH
Then it was the turn of Patrick Gunawan Hartono, composer and student of the Institute for Sonology in the Hague and Suhyun Park, sound designer based in Berlin, forming the Noise Team, presenting a piece entitled 'Spatial Glitsch'. In only a few days time and with no more than a couple hours rehearsal on the system they came up with a remarkably accomplished sounding performance live controlled in the space by Patrick using two iPads. And glitschy it was indeed!
This project intensively explored 4DSOUND's path modules in the 4D.Animator. Paths are created by drawing shapes in a 3d animation of the space. The path's shape can then be stretched, rotated and inverted, while the sound moves along the path at different chosen speeds. The Noise Team made optimal use of this technology by creating at least a dozen of new paths to fit specifically to their sounds, then flipping and turning the movements on-the-fly using spatial controls programmed in Lemur for the iPad.
Here is a good example editing a path in the 4D.Animator (not from Team Noise's project)
KILL THOSE FLIES
This edition of the hacklab again contributed to the growing collection of spatial sound games. Tam Treanor, a sound and interaction designer from Edinburgh, treated us to an entertaining few moments with the interactive game 'Kill Those Flies'.
"After four days coding and very little sleep I made ‘Kill Those Flies’ - a game in which the audience chase an ‘audio fly’ around the space and try to kill it by clapping their hands." said Tam by email after the hacklab. "It was quite a 'buzz' to see people run about and squash those flies during the performance! Now that I’ve got 'the bug' I think I will have to continue the development of this game!"
Treanor realized a Max patch that would localize the approximate position of people by analyzing the pick-up signal of multiple microphones installed throughout the space. The system would know if the clap was approximately close enough to the moving fly to smash it. If the fly died it gave out a recognizable sound of final despair. A simple idea that was not easy to realize, but led to some hilarious and fun action among the audience during the presentation of the Lab. Treanor’s approach to be able to follow the audience through microphone input is an alternative way of real-time position tracking that could be used for many interesting interactions in a spatial sound environment.
TEAM TRICKY FEEDBACK
Perhaps the most impressive ‘hack’ of the 4DSOUND system was realized by Team Tricky Feedback with Timothy Schmele, artist, musicologist, computer scientist associated with the University of music in Karlsruhe, Lichtpiraten, an audio visual collective from Berlin and Andres Perez Lopez from Spain, a c++ programmer, musician and die-hard linux user.
Tricky Feedback started with installing multiple microphones very close to a speaker of the 4DSOUND system. The amount of feedback this would create was regulated by moving virtual sound sources closer or further away from the position of the speaker. Instead of performing a sound input spatially, an inaudible virtual sound moving through space was now used to modulate the amount of feedback the system would produce.
These movements modulating the feedback were live generated by Andres Lopez wearing two MYO sensors on both his arms that responded to muscle tension and the activity and position of his arms moving in space.
Through his system in Supercollider, he fed the control data to the system of Timothy Schmele, who built his own 4DSOUND control patch in Max using OSC connectivity and to the visual system of the Lichtpiraten, real-time transforming different spectral visuals that would be projected onto the space.
Technically advanced and disruptive in its conception, the result of the work surpassed the level of a mere technical achievement but also turned into an emotionally charging, visceral experience. The feedback building up in the system grew and grew, and stabilized itself into a dense cloud of sound. One could literally hear the energy in the space being moved with every twist from Andres's arms and body.
In a compositional context it is very inspiring to explore ways beyond the stereo sound that we're used to. My personal impression on what I've heard is definitely that less is more on 4D, the acoustic space that can be used creatively is an additional element on top of everything else, that should be used wisely.
— Christian Markwart (Lichtpiraten)
It was a real pleasure to host so many inspiring people and their ideas inside the system, and we are already thinking ahead on the next edition of the Spatial Sound Hacklab.
—
Lead image / ONUK
Related:
‘Points on the Curve’ (2016)