SSI



Spatial Sound
Institute


Schema: Towards a Post-Biological Composer (2019)
Tiernan Cross



category:
Essay

 
date:
January 2019


authors:
Tiernan Cross


published by:
Spatial Sound Institute, MONOM and CTM 2019


Tiernan Cross reflects on  the impact of increasingly used artificial intelligence, Big Data and Internet meta-organisms as a process of mediation, extension and transformation of human realities and perceptions through the integration of evermore complex intelligent technology. Analysing the impact of learning processes of computing systems which mimic and integrate user-action behavioural tendencies of human responses, as well as exhibiting the footprint of this on the post-biological time and on future paradigms. Hand in hand he exposes the piece Schema, a dialogue between human decision making and automatised computational systems as an exploration of new ways of sound-making, which he has developed during 2018-2019 at SSI for 4DSOUND System and that revolves around his research.




Without doubt, one of the most dynamic factors of change in our modern lives is the ubiquitous presence of digital technology. The increasingly present role of both telematic and self-reliant systems in mixed reality environments are consequently changing the way in which we experience the world around us, and the ways in which the external world perceives us. Respectively, technology offers us information and sensory experiences that would not be possible without the incorporation of computational systems. For the most part, orthodox technology has run parallel with humanity, automated by some form of human interactivity, control or automation(1). We choose which buttons to press on our smartphones, which codes to execute on our computers and which settings to automate our appliances to. As we make these decisions, smart technologies are learning from us, absorbing and decrypting our actions, preferences and choices from a spectrum of possible outcomes before turning these results into organised data. As such, technological intelligence is becoming increasingly attentive and conversant with the various cruxes of human life. The data-sets built upon the choices we make are exceedingly important, bearing economic, political and cultural consequences. Through the continuous act of digitizing our every move, the computing system as an entity has become smart, and through progressions of analytical forecasting, surveillance and calculation it can predict future user actions, intents and behavioural tendencies. In doing so it can also create a mimicry or hyper-mediation of the user, essentially creating a digital copy of the user’s mind. But what happens when these smart technologies learn too much? Is there a tipping point where smart technologies and cybernetic systems will have gathered too much information?

Large amounts of this information are gathered through network protocols as platforms such as the Internet facilitate the dissemination of one’s self. The Internet as a meta-organism in the current age of Big Data wraps itself around the multifaceted being of humanism, just as much as its impacts, shifts and influences curate proportionate aspects of daily existence. The artificial intelligence and algorithmic capabilities of network protocols have allowed users greater personalised and more individualistic experiences, allowing users to circumnavigate through the excess of the technologically mediated spaces such as the Internet more diligently than ever before. Consequently, network protocols have drastically blurred the boundaries between public and privately information, allowing digital systems to deeply and precisely track the patterns, interrelationships and cycles of human life in a multiplicity of ways. Just as telematic systems have enabled us to source Big Data information from all corners of the globe, they also allow us to intensify and dematerialise our private lives in public spheres, distributing the simulacrum of one’s self across telepresent virtual landscapes of a limitless universal and enduring scale(2).

Setting aside the commercial dictation, economic struggles and political warfare that the era of statistical power brings with it, the age of Big Data is one in which we are entrusting our privacy to the machine in hopes to better orchestrate and optimise our lives. As computational systems increasingly infiltrate and occupy our everyday lives, these systems formulate complex libraries of encrypted psychological, sociological, ethical and aesthetic-based information(3). UCSC-based art historian, Edward Shanken, has argued that telematics have the capacity to “expand perception and awareness by merging human and technological forms of intelligence and consciousness through networked communications”(4). In as much, the term ‘post-biological’ is one used to harness the idea that certain aspects of life are being “mediated, extended or transformed by technology”(5). The implications of such technology can be seen within general cultural evolution, through intelligent technologies exercising reactive and proactive response to situational experiences and through the expansion of human consciousness and the neurological perception of the manifold layers of existence brought about by technologically infused realities. Theorists have continuously questioned the evolution of human consciousness, relaying concepts such as ‘moist-media’ for the post-biological area, questioning the way in which ‘wet’, or organic, biological computing (human neurological structures) intergrade with ‘dry’, or digital computing (technology) and have the capability to enhance and elevate consciousness within the human mind(6). Self-sufficient artificial life is a strong example of a ‘dry’ post-biological organism at the forefront of modern science as self-aware computers or systems are being built to make execute choice. The intelligent constructs of virtual realities, heterotopic spaces and the Internet are examples of ways in which the human mind integrates post-biological technology or non-material systems. As programming systems more so become the norm, so will the technological systems they code become more intelligent, self-sufficient and self-reliant. Dry computing technologies will become increasingly wet as they adapt and learn to mimic the habitual and instinctual tendencies of the human brain.

As of yet, it is the phenomenon of human emotion that is yet to be replicated. We can never experience the phenomenological world of somebody else, as we cannot directly live it. One can live similar perceptual experiences but never precisely replicate or simulate the exact feeling of somebody else’s emotion. Emotion is invaluable in the human brain’s decision making process, leaving a trail of debate regarding the plausibility of computer-generated systems truly making decisions based on emotion and not just statistical data that thus falsely try and mimic emotion. Music and art are prime examples of ways in which external sensory stimuli have the capacity to elicit subjective feelings and emotions in the human neurological system. As money is poured into the development of Artificial Intelligence systems with ‘human emotions’ through data replication in a system’s internal logic(7), it forces us to ask the question: will computers reach a point where they can execute truly subjective feeling of their own, or evaluate personal taste of music through experience? Will artificial intelligence finally figure out emotion and integrate such feeling into action?

Over the course of 2017 and 2018, I developed a project to question ways in which post-biological processes could impact computer music composition. The aim of this project was to create an innovative composition surveying how the exponential convergence of telematic network technologies and audio-visual programming languages could shift paradigms of computer based composition further toward the systematics of post-biologics, with further intent to formulate new, hegemonic spatial aesthetics. The creative output of this research is Schema, a 50-minute fixed-media sound composition designed specifically for 4DSOUND. The work thematically frames the nexus of consciousness between human, computer and object in a post-biological world, exploring sonic relationships between self, nature, and technology. In doing so, this research questions the boundaries of our immediate sound realities and how obscure combinations of auditory information and their elaborate spatialisation can develop new modes of listening experiences. In the sense that digital art often draws notions of immateriality while the manifestation of transduced sound itself has a transient lifespan, the resultant composition of Schema blended sound in a concentrated, spatialised setting.

The conceptual foundation of Schema focussed on building an audio programming system that required no human input, no push of a button, performing tasks without human interaction. A self-sufficient archetype of a recording device that required no automation. The device, built as a standalone application, was up and running for roughly 18 months, without human interaction. It would respond to the input and decibel readings of two virtual audio drivers simultaneously, in which the software device would turn itself on and generate systems to capture mixed combinations of sound before auto-assigning these recordings to memory banks and system folders. The resulting composition of Schema is the aggregation of approximately 2800 sound samples using this software device, which effectively captured physical and telematic field recordings through a post-biological approach. Once recorded, the sound data was then intercepted through a series of patches and systematically allocated to allotted spaces on a MacBook Pro’s SSDrive, building while the patch reschedules itself to perform repeat processes. The resulting raw samples were then either taken back into the standalone application and arranged systematically via computer algorithms; or imported into the audio editing software Ableton Live and arranged musically via human automation. Thus, creating a schism in the compositional process where biological and post-biological creative decisions are diverging.

Spatially, the composition explored ways in which anomalous combinations of digital sound can enhance auditory perception or intrigue a listener. In both theory and practice, the use of technological sound spatialisation exposes the auditory cortex to heavily controlled experiences of sound across time and space. At the root of the biology to be able to hear and interpret sound, human sensitivity, cognition and perception are required, all of which have co-evolved through constant shifts in the technological fabric of our immediate sound environments(8)(9). Previous research has indicated that exposure to abstract sonic experiences offers potential for the human-neurological system to draw new perceptual dexterity such as improved perception of psychoacoustic processes and increased localisation of sound(10). Consequently, it can be hypothesised that persistent exposure to abstract experiences with sound have the capacity to gradually reorder the inveterate dispositions of ‘wet’ biological computing, to reorder the way in which our minds localise, analyse and deconstruct augmented combinations of sound in space. In light of this, Schema experiments heavily with ideas of spatial fragmentation, displacement and signal decorrelation, scattering frequential layers of the same, similar or augmented sounds across space through use of the 4DSOUND system.

When hearing Schema in 4DSOUND, it becomes clear that there are listening intricacies formulated through technological construct that would otherwise be unobtainable in physical settings, with augmented combinations of disintegrated sound in space causing listeners to think twice about the clusters of sound objects that surround them. Spatialisation through 4DSOUND allowed sound objects to be disjointed and held in not only time, but also controlled in Cartesian three-space, offering a mode of listening experiences that can essentially only be attained using digital means. The intention of three-dimensional arrangement was to utilise the precision of the system’s architecture to present concrete sound information from such post-biological systems in atypical, spatialised formations to stimulate cognitive response to unconventional sound localisation. Examples of this are scattered throughout Schema, in which the same identified sound source can be perceived in more than one location at the same time or seemingly different sources merge together into a perceived hybrid entity.

The final version of Schema was composed and recorded entirely at the Spatial Sound Institute in Budapest in mid 2018 and early 2019, morphing mixed reality soundscapes into spatialized concrete sound collages for 42 discrete loudspeakers. Incorporating post-biological theory, the assembly of Schema is segmented into various sections which all fall upon a spectrum from solely human choice and involvement to solely computer automation; meaning some parts of the composition process were recorded using human performance and response, whilst other parts have solely been left up to algorithmic chance, timing and computer-based decisions. This method was used both in composition and for sound spatialisation to play on the paradigms of human-cyber interaction, which resulted in both the human and the computer taking on separate roles of the composer. Various parts of Schema hold a distinct voice of human imperfection, whilst other sections exhibit qualities of quantised mechanisation as aesthetic choices are formulated by processing algorithms. These are recognisable characteristics, contrasts and frictions that begin to form between human and computer, impacting the listening experience of the composition. As much as it may try to eradicate them, the human mind still holds subconscious cultural bounds of creative aesthetic that are adjusted to modern norms in style, mixing, structure and genre. It is clear through Schema that such bounds occupy a part of the human mind during the artistic process, shaping the composition to what it feels is aesthetically appealing. The computer is free of such parameters, using purely algorithmic choice and probability to shape certain parts of the composition, formulating a different set of sound parameters and creating an entirely different listening experience.

Schema is an example of how the convergence of spatial sound technologies and computer programming languages can shift paradigms in experimental and electronic composition, providing a prognostic outlook toward what could become hegemonic aesthetics in everyday listening experiences. Within Schema, such conditions are tested in an experimental context, but as these conditions become more prevalent in our societies they hold the potential to pervade and effect a much wider range of situations in our daily lives. It can be proposed that further exploration into the use of computer programming languages and high density loudspeaker arrays offers potential to expand our understanding of the complex relationships between human, sound and space. Whilst mixed realities and the free-flow of audio-visual hyperdata distributed across post-biological systems are consequently reconfiguring our perceptual experiences of the world(11), complex auditory systems such as 4DSOUND allow these perceptual experiences to be tested even further. It becomes clear that the post-biological and technological infrastructures of the 21st century are revolutionising the landscape and deliverance of sound in ways that could not have been previously anticipated. As a result of these infrastructures and their resulting paradigms in sensory data, our relationship with sound is being rearranged entirely whilst technology as an artistic tool offers immense systematic growth in supporting human expression in creativity. Considering the rapid rate of computer-based progression and the potential free agency of artificial intelligence, it becomes important to think about the future of the human mind and its role in creativity, to question ways in which human consciousness could become redundant to artificial systems.

The future of post-biological systems in music is almost guaranteed. Schema as a whole is nothing more than a combination of frequency outputs formulated through a series of both human and computer decision-making. Whilst computing systems have yet to master human emotion, the boundaries of sound composition are practically non-existent. It could become increasingly difficult for a listener to determine what was creatively human driven and what is purely algorithmic. Whilst artificial systems have not grasped emotive response and its application in music, they are able to mimic human action, indicating a probability that they will be able to learn so much that their actions might give the facade of emotive response.




About the Author

Tiernan Cross is a composer, sound artist and researcher based in Sydney, Australia. As the current recipient of the University of Sydney’s Eleanor Dunne Scholarship, Tiernan is completing postgraduate research through the Sydney Conservatorium of Music, focusing on neurological conditioning, composition and post-biological sound aesthetics.

His research and compositional practice questions the relationships between the evolution of perceptual encounters and ways in which we as humans neurologically process complex combinations of sound in mixed-reality environments. Classically trained through the Sydney Conservatorium of Music, Australian Film, Television & Radio School’s postgraduate program and the Sydney School of Architecture, Tiernan’s works have been exhibited and performed in China, France, Belgium, Spain, Canada, USA, Australia, Germany and Japan. Tiernan also holds a Bachelor’s Degree in Communication from Griffith University.




References

1. Rossow, A. (2018). Artificial Intelligence and Smart Technology Is Bringing Convenience To A Whole New Level. Retrieved from https://www.forbes.com
2. Ascott, R. (2006). Introduction. In R. Ascott (Ed.), Engineering nature : art & consciousness in the post-biological era. Bristol, UK: Intellect, pp. 9-10.
3. Berry, D. M., & Dieter, M. (2015). Postdigital aesthetics: art, computation and design. Basingstoke: Palgrave Macmillan.
4. Ascott, R. and Shanken, E. A. (2003). Telematic Embrace: Visionary Theories of Art, Technology and Consciousness, Berkeley, CA: University of California Press, pp. 6.
5. Ascott, R. (1998). Consciousness Reframed: Art and consciousness in the post‐biological era. Digital Creativity, 9(1), pp. 5-6.
6. Ascott, R. and Shanken, E. A. (2003). Telematic Embrace: Visionary Theories of Art, Technology and Consciousness, Berkeley, CA: University of California Press, pp. 1.
7. Darlington, K. (2018). AI Systems Dealing with Human Emotions. Retrieved from https://www.bbvaopenmind.com/en/ai-systems-dealing-with-human-emotions/.
8. Borgo, D. (2013). ‘Beyond performance transmusicking in cyberspace’, in N. Cook and R. Pettengill (eds), Taking It to the Bridge, Ann Arbor, MI: University of Michigan Press, pp. 319–348.
9. Sterne, J. (2003), The Audible Past: Cultural Origins of Sound Reproduction, Durham, NC: Duke University Press.
10. Cross, T. (2018). Neurological perception and sound-based creativity in post-biological realities: Recontextualizing reflective practice for technoetic environments. Technoetic Arts: A Journal of Speculative Research, 16(1), Bristol, UK: Intellect, pp. 23-31.
11. Stenslie, S. (2006). ‘Symbiotic interactivity in multisensory environments’, in R. Ascott (ed.), Engineering Nature: Art and Consciousness in the Post-Biological Era, Bristol, UK: Intellect, pp. 153–158.





Related:

Psychoacoustics: An Introduction (2014) [PUBLICATION]