On 4DSOUND (2011)
Paul Oomen
This essay was originally delivered as a spoken lecture during the Symposium Spatial Music in September 2011, as part of the International Week of Contemporary Music.
Oomen reviews his diverse endeavours in composing spatial music, leading up to the creation of 4DSOUND technology, a development that has been central in his work since 2007. The main concerns of this development are introduced by four reflections on spatial elements: space, distance, movement and perspective.
On Space
As long as I remember, space has always been a main source of inspiration and a model for my compositions. Space concerns with both the tactile sensation of physical presence and the ephemeral awareness of atmosphere. Space also concerns with social organization, human behaviour and interaction. Imagining sounds spatially guides my musical choices. It provides me with a dramaturgy for the appearance of sounds in a musical discourse. How sounds changes in time equals to how I imagine them moving in space.
In my earlier works, still within the borders of ‘regular’ composed music for instruments and ensembles, my imagination of space mainly expressed by means of contrasting dynamics in the music. When sounds decrease in intensity they suggest a movement away, when loudness intensifies they are suggested to come closer. A rather simplistic approach, that nevertheless taught me some essential awareness about spatiality. The need to make the listener really feel a sound piercing through one's head urged me to go into extremes with dynamics. To sharpen musical contrasts as much as possible increases the listeners awareness of suggested depths in spatial distances.
In my opera Kane, one soprano evokes a multitude of voices by performing sharp cut vocal fragments with contrasting intensities and speeds. As one line is whispered and the next word screamed, as one phrase gains intensity and the other fades away, the polarities of the voice unfold spatial distances. Yet at the same time they embody the presence of different dramatic expressions. Distinct patterns of behaviour equal specific spatial forms, as retreating is backwards, attacking is forward, being close allows intimacy, taking distance suggests leaving, swaying left to right is indecisive.
In this case spatial experiences are mere scripts that lead to musical gesture and form. Later I became more and more interested in the real thing. Not representing space but the actuality of space surrounding performer and listener.
In my work from 2006 on I have turned towards an experience-based site-specific art that takes place in the outside world, instead of the safe inside of the concert hall. The environment of a musical performance, whether in nature, urban surrounding or cyberspace, becomes itself the main subject of musical composition. Not an idea of space represented by sound, but the actual physical presence of the listeners and sounds in space is what is musically explored.
My opera Water was situated in many different spaces of the sewage plant in the neighborhood Ondiep in Utrecht. With the composition of this opera I wanted to acknowledge first of all the magnificent presence of noises and tones in the industrial spaces, resulting from machines running and water flowing. With my music I extended these sounds and amplified them. By adding instrumental and electronic sounds I attempted to make the listener more aware of the physical proportions of the extraordinary spaces and focus attention to details inside the spaces that would otherwise stay unnoticed. I wanted to express atmospheres and emphasize the mood of each individual space more articulated.
In my opera These Fragments I have shored against my ruins space became not only a sounding subject of the work, but also a place for human interaction. The audience, confronted with the intimacy and seduction of touch by the singers, is triggered to start moving, either attracted by or apprehensive for the physical encounter with the performers. By inducing movement of the listener, the musical space is constantly explored from different perspectives.
With the development of 4DSOUND, and parallel to this the opera Nikola inspired on the life, work and thought of Nikola Tesla, my spatial exploration have once again evolved to a next level. Now, the actuality of the space performers and listeners are in transcends to an illusionary level. The actual space is replaced with or interfered by the lifelike experience of transforming virtual spaces.
On Distance
I imagine the most elementary state of our sensory experience. The beginning of our lives during the embryonic phase in the womb. Our ears are not yet ears and do not hear. Our body is surrounded by water and knows no orientation. Sounds move all around us, we cannot hear, but vibrations touch our body. Differences in intensity turn into a sense of distance and proximity. Spatial experience is born. Although we seem to experience sound through our ears, this observation shows that on a fundamental level, sound first appeals to physical sense.
It is clear that sound waves are physically present in space and therefore interact not exclusively with the ears, but with the entire body. Nevertheless, in general listening situations physical sense of sound is only perceived subconsciously. The ears are privileged to make us aware of sound because of the close neural connection between the ears and the brain. Only under extreme circumstances one will experience sound physically instead of cerebrally. One has to be literally shaken to become aware of this dimension of sound.
Interesting enough, 4DSOUND enhances a strong awareness of physicality, not by means of extreme volumes but by evoking a physical phantom. The source that causes the sound, which is not there but virtually reproduced by the 4DSOUND system, has to be cerebrally re-assured. If we really hear something, it has to be true even though we know it is only an illusion. Therefore sound is not only listened to. The physical source that produces the sound is felt to be present as well. Through this involuntary synaesthetic detour, 4DSOUND has the potential to enlarge the experience of listening with a new dimension – the conscious awakening of physical sense.
Physicality of sound has everything to do with distance, or in fact being able to overcome distance and make the listener experience the phantom source piercing right through the body. For this we have to make the listener forget about the loudspeaker, since localizing the actual sound source will always disturb the physical interaction with the phantom source.
For a new approach of this problem we considered that one could overrule the interaural cross correlation, which is the mechanism behind the listener’s ability to localize any non-virtual sound sources, i.e. the loudspeakers themselves. The resulting application for contiguous phantom imaging, patented by Leo de Klerk in 2010, describes a sound system that produces coherent interaction in the vertical and horizontal plane in order to produce unlimited spatial images that cannot monaurally nor binaurally be decomposed to their root sources, i.e. the actual loudspeaker drivers. The loudspeaker becomes audibly non-localizable. As a consequence the listening area is not anymore restricted to a ‘sweetspot’, the listener can walk in and outside the speaker array and appreciate the same image. One can physically explore 3-dimensional sonic shapes in space, can walk around them, virtually touch them and walk through them.
As a consequence, in 4DSOUND one does not only listen to sound spatially, but experiences sound as an appearance of physical energy.
On Movement
An important example for me referring to a physical and energetic experience of sound in space is techno music. In techno, physical sense is present due to extreme volumes and frequencies that fill the space and make the listener lose orientation. One does not listen to music, but is positioned physically inside the music itself. A techno party is a thrilling atmosphere of sounds, lights and bodies that the listener is part of. It is not possible to escape the movements of energy that take place in the music. One does not listen to techno, but feels it.
In techno, all musical actions are supportive of an energetic process. To charge, conduct, explode and disperse energies inside the body of the listeners. The compositional models of techno music resemble in a way the spatial processes of 4DSOUND. It is about transformations of dimensions and intensities. A play of constant shifts in spectrum and phase of the sounds that make the listener lose orientation and transcend the actuality of the space one is in.
In a recent interview, techno artist Richie Hawtin referred to the energetic process of a techno party as 'radical contingency'. A state where things are as yet undefined, which makes all decisions contingent and forces the imagination into action. Being constantly at the edge of creating a moment which everybody is waiting for, a moment that everybody is wishing to become clear and become defined, but as soon as it is, looking beyond it again.
I have always imagined that a spatial sound system should be performed in such way. Like a DJ behind his decks, buttons and slides one should be able to perform space. Spatial experience as a reciprocal relation between musician and listener. The musician is controlling a sounding environment influencing the listener’s physical movement. Yet at the same time being influenced by and reacting to the listener’s movement behaviour. The listener is seduced to explore the sonic surrounding. Dependent on ears and body, the listener has to discover space and becomes part of an encompassing experience that can only be discovered step by step, and never overviewed in totality.
From a listener’s point of view, it is essential that such reciprocality with the sounding space is restored. One of my intentions behind the development of 4DSOUND has been to let the audience move freely and react spontaneously to the spatial experience they are in. And therefore restoring play and interaction as a vital dimension of a lifelike spatiality. In daily life, we are exposed to continuous movement in the environment around, above and beneath us. And in this environment we move ourselves in complex patterns, sometimes fast and straight, then slow and hesitant, turning back and forth, possibly laying, standing, bending or sitting. All these movements have a grave effect on our listening experience. We are continuously affected by changes of movement surrounding us, but equally, by our patterns of behaviour we influence our perception of this environment and stimulate the movement in the environment itself.
The appearance of sound as perceived by the listener are inseparable from their spatiality. Rhythmic and pitched intervals are determined by direction and speed of movement, timbres of sounds are gravely influences by distance, elevation and angle of their spatial position, volume and intensity of sounds depend entirely on the degree of distance towards the perceiver. All spatial actions have a dramatic impact on the perceived sounding result. Sound can be transformed entirely by the speed, distance, elevation and direction they are moving in.
Our attention has been focused on translating such spatiality into a musical creative process. With the instrumental design of 4DSOUND we have focused on making every aspect of spatiality and spatial movement controllable to the smallest detail, but also take a step aside from a too naturalistic or scientific approach on reproducing spatiality.
From a musical point of view, it is entirely dependent on the musical context whether and which approach to spatiality makes sense. In the first place, it is a musical question to whether spatial attributes should be enlarged, diminished or even bypassed to be convincing, not a technological one. It is also doubtful whether a ‘correct’ natural appearance of spatial sound can be captured in any fixed virtual setting, as a bird flying in the sky demands very different properties of distance, spectrum and elevation than if an airplane flies over. In most cases, it is better to dispose of a broad set of techniques and use the hearing to make a setting that truly works musically.
4DSOUND has been created to meet such demands, to offer all possible instrumental possibilities to create lifelike spatiality, but use them modular, on demand and as part of a live musical performance.
On Perspective
Our general awareness of spatiality is strongly dependent on the omnidirectional and non-selective reception of our hearing. While the eyes are conditioned for complex rendering of objects and their characteristics, the hearing primarily follows continuity in time and space. Therefore the ears detect with great accuracy the slightest changes in speed, direction, distance and voluminous proportions between ourselves and the environment. More than any of the senses, the specificity of hearing is vital to experience spatiality.
In early Renaissance paintings, perspective itself, or a fascination of the possibility to represent perspective, is often the central theme of the image. In reconstructing the natural deviation of angles that follows from the limited curve of the retina, painters were able to suggest a dimension of the visual experience vital to everyday perception. Reconstructing spatial perspectives for the ears means expanding the image to a full panoramic scope, as the ears do not cancel out angles but always discern space omnidirectional. And similar to the movement of a camera in film, we do not only perceive a surrounding from a still point of view, but we also move ourselves through this space and therewith change the scope of our perspective.
When I started with 4DSOUND I was always imagining the space as a surrounding where external entities and energies where present and moving. While developing and discovering 4DSOUND further, I became more and more interested in the interior experience of space, the way we position our body and therewith create perspectives of spatiality.
Such interior experience of space are based on elementary motions of the body like moving forward and backward, turning around, moving up, falling down and being still. These physical motions gain yet another dimension when their extremes are being tweaked. One can think of moving forward at rocket speed, or falling down for kilometers with increasing speed and so on. Also, more delicate physical experiences can be imagined such as experiences of dizziness, shivering, shaking or scatter. In other words, affections that come from within the body and blur the totality of the spatial perception.
With this idea we developed the ‘global transform’ module for 4DSOUND. It is basically the reconstruction of a virtual listener in space. Global transformations makes all the sounds in space move proportional to the direction and position of a virtual listeners body. This virtual listener in a way replaces the actual listening position of each individual listener in space, makes believe that we experience the world from this perspective.
When moving on the highway or in a train, the perspective of the surrounding landscape transforms proportional to the speed we move with. the closer the object, the faster it moves past us. spatial distances become rhythmic speeds upon a basic tempo that is the speed we virtually move in. When rotating around, all sound globally receive a position within the rotation. Static sounds rotate proportional around the virtual listener. Straight moving sounds become spirals, Rotating sounds are either accelerated or diminished by their direction of movement. One can imagine rotations to achieve very high speeds and become tones, forming harmonic relations with each, like revolving sounding bodies.
The perspective of spatial listening gains yet another surreal highlight when the size of a space is globally transformed. A sounding space can diminish to the size of a grain of sand, and then expand again to cosmic proportions. By continually expanding the size of a virtual space, sounds can continue to evolve from within a Pandora’s box, ever-expanding illusions of a sound captured within a sound within a sound.
—
Related:
‘4DSOUND: A New Approach to Spatial Sound Reproduction & Synthesis’ (2012-2016)
‘4DSOUND: A New Dimension’ (2014)