What The Future Sounds Like (2018)
William Ralston
category:
Interview
date:
September 2018
authors:
William Ralston
published by:
Hole & Corner
Interview
date:
September 2018
authors:
William Ralston
published by:
Hole & Corner
Arguably the world’s first omnidirectional sound environment, 4DSOUND offers the chance to experience individual recorded sounds as independent physical entities in space. Over ten years in development, its founder Paul Oomen says he is only just beginning to understand its implications…
Situated in the east of Berlin, on the banks of the Spree, is Funkhaus, an enormous multi-purpose campus with a long and cultured history. Built in 1950 as the broadcasting base for GDR’s state radio, it’s a peculiarly chilling and rather desolate spot to visit nowadays. Head out of the main hall – itself home to a plethora of musical studios, performance rooms and offices – and you’ll find an assortment of weathered warehouse spaces, the majority of which are unoccupied. But behind one of those large industrial iron doors is the world’s first ever 4DSOUND system, the most advanced spatial sound instrument of its kind. The only other one of this scale can be found at the Spatial Sound Institute in Budapest, Hungary.
The system itself is an elegant and sophisticated construction - a product of over a decade of dedicated research and development by Paul Oomen and his team of engineers. Each system is tailored to the space, but this particular one consists of 16 industrial metal cylinders, each about a foot in diameter and around 12 feet tall, distributed in a four-by-four grid on an acoustically transparent platform no bigger than a regular dancefloor. The room itself is just a little larger. The floor is grated for the purposes of the nine sub-woofers below. Inside each of these cylinders are bundles of speakers positioned at three heights: below feet, head height, and overhead. Off to one side is a large workstation, like a DJ booth of the future, with a desk, several monitors, and amplifiers.
You’d be forgiven for not being familiar with 4DSOUND or Paul Oomen, the man behind it all. Oomen began his research in 2007, intent on creating an integrated software and hardware system that provides an omnidirectional sound environment. In more simple terms, this means conceiving a fully immersive space where sounds appear as independent physical entities. This allows listeners to actually locate sounds and walk over to them. It’s something that must be experienced to be properly understood. And despite having spent a large portion of his adult life developing the system, Oomen explains that understanding the full capabilities of his creation remains a work in progress.
Oomen spent much of his youth in theatre performance. Born and raised in Amsterdam, he began acting aged just 14 and enjoyed moderate success, performing in various local productions. In parallel with this was a burgeoning interest in music. Oomen recalls ‘scribbling his own compositions’ aged just seven or eight, albeit with an invented form of notation. Aged 18, he enrolled at Amsterdam’s Music Conservatory, and later Berlin’s University of the Arts, where he studied for a further two years.
It was during this period of study that Oomen’s interests began to merge. Though compelling, his studies of classical music felt like ‘a suit that never quite fitted,’ he says. His main focus was writing music for theatre, and his tutors encouraged him to produce his own experimental sound-based pieces and accept any composition commissions that came his way. It soon became clear that his most striking pieces of work were all composed with a particular space in mind, be it virtual or absolute, thus unearthing a fascination with spatial relationships. ‘It was always about more than just music,’ he says. ‘I would never just imagine a melody; I would always imagine the space where the piece would be performed.’
One natural progression of his interest was site-specific theatre, where the setting is integral to the production. Oomen had long wished to break through the idea of the spectator as passive in the theatrical experience, and his growing fascination for the dynamics of sound within space inspired him to take action. After completing a number of site-specific theatre works, Oomen had grown frustrated that sound always inhibited complete audience immersion. ‘I tried hiding speakers in corners and also using different sound systems,’ he recalls, ‘but from a composition perspective, I felt extremely limited.’ Only with a more advanced sound system could he properly sculpt the space that the audience and performers were in. ‘I just didn’t have the tools available,’ he explains. And so he started to develop them.
An important catalyst in this decision was Oomen’s fascination with Nikola Tesla, the Serbian-American physicist. Oomen had become enchanted with Tesla’s discoveries surrounding movement and energy, and wished to compose a work about it – an ‘extremely ambitious idea,’ as he explains it. The essence of the piece was to use sound to show how energy moves through space; and to do so, he needed a sonic instrument that was much more advanced than anything already available. ‘I wanted people to be able to locate sounds around them and physically feel sounds moving within the space,’ Oomen explains. Drawing on his knowledge and that of those around him, Oomen sketched out an initial draft system and began his research.
But it wasn’t going to be easy. Oomen’s obstacles could be assorted into two broader categories: finding a capable speaker system (and configuring a layout); and coding a sufficiently advanced software interface to allow him the control over the sounds.
Oomen’s first step was to consult some talented friends in the Music Technology department of Utrecht’s Academy of Arts. He detailed his intentions before hiring a team of programmers: Poul Holleman, Luc van Weelden, and Salvador Breedwho (all of whom remain involved). After just 12 months in development, the team had settled upon the software and conducted their first tests using a grid of 27 regular PA speakers. It worked ‘surprisingly well,’ Oomen recalls, but there was much more work to be done. For instance: traditional PA speakers point at the listener, and so the angle of projection is narrow. But for the sound to feel like it was coming from everywhere, Oomen needed speakers that projected much more widely.
But there weren’t too many options available. Research in these realms was very limited back then; the most advanced developments were in academic contexts and not suited for creative application. In light of this, Oomen started to design his own speaker that would disperse sound equally over 180 degrees. Progress was made, but his work was cut short when he connected with Leo de Klerk, founder of Bloomline Acoustics. De Klerk, another leading spatial sound enthusiast, had designed the world’s first fully omnidirectional speaker (ie one that radiates sound in all directions) and was preparing to patent his invention. It was more advanced than anything Oomen had developed, and so he adopted it as one his system’s main components. After saving up for more than a year, in 2010 Oomen placed an order for 48 omnidirectional speakers. ‘It was the missing piece of our jigsaw,’ he explains.
With speakers in hand, in 2011 Oomen and the team rented a large Amsterdam warehouse to begin configuring the system. The plan was to distribute the speakers equally through the space in the same rectangular grid as before, although various experts had anticipated that the balance of sound would never be quite right. Nonetheless, Oomen proceeded and was ‘pleasantly surprised’ by the result. The ensuing period was then spent refining the technologies and learning how to use them. ‘It was purely intuitive,’ Oomen explains, ‘and involved a lot of trial and error.’ After a year of dedicated research and development, Oomen presented the Tesla production, titled NIKOLA, in September 2012, which he describes as ‘the most nerve-wracking thing’ he’s ever done.
By this point, however, Oomen’s eyes had moved onto bigger things. The Tesla production’s success highlighted a wealth of new creative opportunities, few of which even Oomen could have anticipated. Rather than just a tool for composing sounds and adding new textures, he had created an entirely new instrument in itself – one that allows artists to explore new performative disciplines and express their ideas with a new dimensionality. ‘Only then did it become clear just how much potential the system has,’ he explains, ‘and I wanted to begin exploring what was really possible.’ Interestingly, his explorations have long been privately funded; Oomen accepted some support for the Tesla production but cut all ties thereafter in favour of financial independence and operational flexibility. ‘At this point, I was completely obsessed with the idea,’ he explains. ‘I had some capital myself that I had made with earlier productions – and I invested all of that in the further development.’ Any additional outlays have been covered through bank loans.
The system entered the more conventional electronic music world in October 2012 at Amsterdam Dance Event, where it featured music composed by Peter Van Hoesen. There have since been numerous other artist showcases, but this has given rise to its own obstacles. The system requires artists to learn an entirely different medium of communication, like a whole new language, if you will. ‘Artists have to begin by exploring, sound-by-sound, what you can do with the system,’ Oomen explains. ‘It’s an instrument and it requires you to learn it as such.’ For these reasons, Oomen is particularly excited about working with young artists rather than established ones. ‘If you want to make a meaningful production you have to start from scratch,’ he says. ‘Established artists can deliver great content but they already have an ingrained approach for sound.’
Questions must now be asked about where 4DSOUND can and will go. The technology has advanced well beyond our understanding of it; there isn’t anyone in the world proficient enough to exploit even a modicum of its capacity. ‘When people learn the piano there are centuries of tradition on how to cultivate skills on such an instrument,’ Oomen explains. ‘There is a interesting research into the exploration of space, but we are really dealing with an entirely new instrument,’ he continues. ‘It’ll take decades for us to properly comprehend the possibilities.’
Nonetheless, Oomen is convinced that it will never take shape in a club context. Not only does it require an entirely different way of performing, much more than just playing records, but it also relies heavily on the listener. Spatial sound requires near silence so as to allow sounds to drift to the edge of perception and return; the experience will never work in situations where people are used to talking. It’s similar to a cinema: attendees must abide by certain conditions to maximise the experience. ‘This doesn’t exist with spatial sound yet,’ Oomen explains. ‘Instead of bringing the system into contexts where there is a regular way of listening, we need to create a new context, or a new form of presentation, to make this really come to life.’
Important in this endeavour is Monom, the Berlin-based collective headed up by Australian William Russell. With the help of an angel investor, Zak Khutoretsky (DVS1), Russell purchased the original system and installed it at Funkhaus in July 2017. Much of the focus since then has been on electronic music production and performance, whereby established producers spend some weeks learning the system before performing in front of a live audience - but “this is for any artist to explore,” Russell explains. Classical musicians have already utilised the system to great effect. “It’s not just about music; it’s about emotions and harnessing the way we have evolved to perceive sound and space, harnessing emotional responses inscribed in our ancient memories, to deliver us to new states,” Russell adds. There are plans in place to explore classical and transcendental applications such as healing, and meditation. Meanwhile, Oomen and his team at Budapest’s Spatial Sound Institute are focusing on broader applications of spatial sound outside of music, including scientific, architectural, and in media design.
These are ‘extremely exciting’ times, Oomen explains. A growing global interest in spatial sound has facilitated developments and stabilised the company from a financial perspective. The team will also soon present some more mobile and flexible systems that will, of course, allow them to demonstrate the system much more readily. 4DSOUND has already started supplying various installations, not all of which are omnidirectional, using the software and the approach as tools to provide auditory experiences.
For now, the journey continues as interest grows. ‘We have built an extensive instrument that people are just beginning to explore,’ Oomen says. ‘It’s a huge development in the tools that artists can use to express themselves.’ And it’s quite possible that our understanding of it will only become clear in the many decades to come.
—
The system itself is an elegant and sophisticated construction - a product of over a decade of dedicated research and development by Paul Oomen and his team of engineers. Each system is tailored to the space, but this particular one consists of 16 industrial metal cylinders, each about a foot in diameter and around 12 feet tall, distributed in a four-by-four grid on an acoustically transparent platform no bigger than a regular dancefloor. The room itself is just a little larger. The floor is grated for the purposes of the nine sub-woofers below. Inside each of these cylinders are bundles of speakers positioned at three heights: below feet, head height, and overhead. Off to one side is a large workstation, like a DJ booth of the future, with a desk, several monitors, and amplifiers.
You’d be forgiven for not being familiar with 4DSOUND or Paul Oomen, the man behind it all. Oomen began his research in 2007, intent on creating an integrated software and hardware system that provides an omnidirectional sound environment. In more simple terms, this means conceiving a fully immersive space where sounds appear as independent physical entities. This allows listeners to actually locate sounds and walk over to them. It’s something that must be experienced to be properly understood. And despite having spent a large portion of his adult life developing the system, Oomen explains that understanding the full capabilities of his creation remains a work in progress.
Oomen spent much of his youth in theatre performance. Born and raised in Amsterdam, he began acting aged just 14 and enjoyed moderate success, performing in various local productions. In parallel with this was a burgeoning interest in music. Oomen recalls ‘scribbling his own compositions’ aged just seven or eight, albeit with an invented form of notation. Aged 18, he enrolled at Amsterdam’s Music Conservatory, and later Berlin’s University of the Arts, where he studied for a further two years.
It was during this period of study that Oomen’s interests began to merge. Though compelling, his studies of classical music felt like ‘a suit that never quite fitted,’ he says. His main focus was writing music for theatre, and his tutors encouraged him to produce his own experimental sound-based pieces and accept any composition commissions that came his way. It soon became clear that his most striking pieces of work were all composed with a particular space in mind, be it virtual or absolute, thus unearthing a fascination with spatial relationships. ‘It was always about more than just music,’ he says. ‘I would never just imagine a melody; I would always imagine the space where the piece would be performed.’
One natural progression of his interest was site-specific theatre, where the setting is integral to the production. Oomen had long wished to break through the idea of the spectator as passive in the theatrical experience, and his growing fascination for the dynamics of sound within space inspired him to take action. After completing a number of site-specific theatre works, Oomen had grown frustrated that sound always inhibited complete audience immersion. ‘I tried hiding speakers in corners and also using different sound systems,’ he recalls, ‘but from a composition perspective, I felt extremely limited.’ Only with a more advanced sound system could he properly sculpt the space that the audience and performers were in. ‘I just didn’t have the tools available,’ he explains. And so he started to develop them.
An important catalyst in this decision was Oomen’s fascination with Nikola Tesla, the Serbian-American physicist. Oomen had become enchanted with Tesla’s discoveries surrounding movement and energy, and wished to compose a work about it – an ‘extremely ambitious idea,’ as he explains it. The essence of the piece was to use sound to show how energy moves through space; and to do so, he needed a sonic instrument that was much more advanced than anything already available. ‘I wanted people to be able to locate sounds around them and physically feel sounds moving within the space,’ Oomen explains. Drawing on his knowledge and that of those around him, Oomen sketched out an initial draft system and began his research.
But it wasn’t going to be easy. Oomen’s obstacles could be assorted into two broader categories: finding a capable speaker system (and configuring a layout); and coding a sufficiently advanced software interface to allow him the control over the sounds.
Oomen’s first step was to consult some talented friends in the Music Technology department of Utrecht’s Academy of Arts. He detailed his intentions before hiring a team of programmers: Poul Holleman, Luc van Weelden, and Salvador Breedwho (all of whom remain involved). After just 12 months in development, the team had settled upon the software and conducted their first tests using a grid of 27 regular PA speakers. It worked ‘surprisingly well,’ Oomen recalls, but there was much more work to be done. For instance: traditional PA speakers point at the listener, and so the angle of projection is narrow. But for the sound to feel like it was coming from everywhere, Oomen needed speakers that projected much more widely.
But there weren’t too many options available. Research in these realms was very limited back then; the most advanced developments were in academic contexts and not suited for creative application. In light of this, Oomen started to design his own speaker that would disperse sound equally over 180 degrees. Progress was made, but his work was cut short when he connected with Leo de Klerk, founder of Bloomline Acoustics. De Klerk, another leading spatial sound enthusiast, had designed the world’s first fully omnidirectional speaker (ie one that radiates sound in all directions) and was preparing to patent his invention. It was more advanced than anything Oomen had developed, and so he adopted it as one his system’s main components. After saving up for more than a year, in 2010 Oomen placed an order for 48 omnidirectional speakers. ‘It was the missing piece of our jigsaw,’ he explains.
With speakers in hand, in 2011 Oomen and the team rented a large Amsterdam warehouse to begin configuring the system. The plan was to distribute the speakers equally through the space in the same rectangular grid as before, although various experts had anticipated that the balance of sound would never be quite right. Nonetheless, Oomen proceeded and was ‘pleasantly surprised’ by the result. The ensuing period was then spent refining the technologies and learning how to use them. ‘It was purely intuitive,’ Oomen explains, ‘and involved a lot of trial and error.’ After a year of dedicated research and development, Oomen presented the Tesla production, titled NIKOLA, in September 2012, which he describes as ‘the most nerve-wracking thing’ he’s ever done.
By this point, however, Oomen’s eyes had moved onto bigger things. The Tesla production’s success highlighted a wealth of new creative opportunities, few of which even Oomen could have anticipated. Rather than just a tool for composing sounds and adding new textures, he had created an entirely new instrument in itself – one that allows artists to explore new performative disciplines and express their ideas with a new dimensionality. ‘Only then did it become clear just how much potential the system has,’ he explains, ‘and I wanted to begin exploring what was really possible.’ Interestingly, his explorations have long been privately funded; Oomen accepted some support for the Tesla production but cut all ties thereafter in favour of financial independence and operational flexibility. ‘At this point, I was completely obsessed with the idea,’ he explains. ‘I had some capital myself that I had made with earlier productions – and I invested all of that in the further development.’ Any additional outlays have been covered through bank loans.
The system entered the more conventional electronic music world in October 2012 at Amsterdam Dance Event, where it featured music composed by Peter Van Hoesen. There have since been numerous other artist showcases, but this has given rise to its own obstacles. The system requires artists to learn an entirely different medium of communication, like a whole new language, if you will. ‘Artists have to begin by exploring, sound-by-sound, what you can do with the system,’ Oomen explains. ‘It’s an instrument and it requires you to learn it as such.’ For these reasons, Oomen is particularly excited about working with young artists rather than established ones. ‘If you want to make a meaningful production you have to start from scratch,’ he says. ‘Established artists can deliver great content but they already have an ingrained approach for sound.’
Questions must now be asked about where 4DSOUND can and will go. The technology has advanced well beyond our understanding of it; there isn’t anyone in the world proficient enough to exploit even a modicum of its capacity. ‘When people learn the piano there are centuries of tradition on how to cultivate skills on such an instrument,’ Oomen explains. ‘There is a interesting research into the exploration of space, but we are really dealing with an entirely new instrument,’ he continues. ‘It’ll take decades for us to properly comprehend the possibilities.’
Nonetheless, Oomen is convinced that it will never take shape in a club context. Not only does it require an entirely different way of performing, much more than just playing records, but it also relies heavily on the listener. Spatial sound requires near silence so as to allow sounds to drift to the edge of perception and return; the experience will never work in situations where people are used to talking. It’s similar to a cinema: attendees must abide by certain conditions to maximise the experience. ‘This doesn’t exist with spatial sound yet,’ Oomen explains. ‘Instead of bringing the system into contexts where there is a regular way of listening, we need to create a new context, or a new form of presentation, to make this really come to life.’
Important in this endeavour is Monom, the Berlin-based collective headed up by Australian William Russell. With the help of an angel investor, Zak Khutoretsky (DVS1), Russell purchased the original system and installed it at Funkhaus in July 2017. Much of the focus since then has been on electronic music production and performance, whereby established producers spend some weeks learning the system before performing in front of a live audience - but “this is for any artist to explore,” Russell explains. Classical musicians have already utilised the system to great effect. “It’s not just about music; it’s about emotions and harnessing the way we have evolved to perceive sound and space, harnessing emotional responses inscribed in our ancient memories, to deliver us to new states,” Russell adds. There are plans in place to explore classical and transcendental applications such as healing, and meditation. Meanwhile, Oomen and his team at Budapest’s Spatial Sound Institute are focusing on broader applications of spatial sound outside of music, including scientific, architectural, and in media design.
These are ‘extremely exciting’ times, Oomen explains. A growing global interest in spatial sound has facilitated developments and stabilised the company from a financial perspective. The team will also soon present some more mobile and flexible systems that will, of course, allow them to demonstrate the system much more readily. 4DSOUND has already started supplying various installations, not all of which are omnidirectional, using the software and the approach as tools to provide auditory experiences.
For now, the journey continues as interest grows. ‘We have built an extensive instrument that people are just beginning to explore,’ Oomen says. ‘It’s a huge development in the tools that artists can use to express themselves.’ And it’s quite possible that our understanding of it will only become clear in the many decades to come.
—
Related:
