Synaesthesia, the neurological condition of cross-modal association between the senses, has recently re-emerged as a central concern in the audio-visual arts. The recent ‘Visual Music’ exhibition (Los Angeles MOMA) offered an illuminating look into the history of sound and image convergence dating as far back as the early 19th century. Originally, the primary province of synaesthetic enquiry resided with musicians. Arguments over which pitch evoked which corresponding colour were rife and over time it became clear in both the arts and sciences that there was no hard and fast rule; synaesthesia was inherently subjective with very little chance of overlap between individual experiences. Today, with most attempts to produce a synaesthetic outcome (by manufacturing some link between the sound and visual materials) mediated through various technologies, the different mappings of each technological set-up, mirrors the subjectivity of actual synaesthetic experience
Image Robin Fox
Working with the confluence between sound and light based electricities, it is possible to describe the abstracted geometry of a sound visually, in both two and three dimensional space. Since 2003 I have been engaged with exploring two dimensional environments using the relatively simple set-up of stereo audio signal sent to a Cathode Ray Oscilloscope (CRO) with the left audio channel activating the y axis and the right audio channel controlling the x axis. Working in this way, the same signal that causes the speaker cones to vibrate (hence producing sound) also activates the image. This audio-visual signal equivalence goes beyond the mere syncing of sound and image and tends more toward a direct causal relationship between what you hear and what you see. Interestingly, as the CRO only produces a single green light beam that dances across a phospherous screen, the relationship eschews notions of sound to colour correspondence focussing instead on the relationship between sound, shape and movement. In exploring these relationships I discovered relatively quickly that sounds with a harmonic spectrum tended toward messy and uninteresting shapes whereas inharmonic sounds generated an angularity and geometric intensity (almost a sense of order) that was much more appealing. Over time I developed a lexicon of sound to visual relationships; the early results of which are documented on the ‘backscatter’DVD, released in 2004 on Synaesthesia records.
Performing extensively, by focussing a camera onto the small CRO screen and projecting the image live, I became increasingly frustrated with the loss of signal resolution and the corresponding loss of depth to the image. I also became acutely aware of the limitations of viewer perspective and the general ‘inactivity’ of most presentation environments. These limitations have moved the work beyond the proscenium, exploring the possibilities of activated (in some cases ‘mirrored’) spaces that are either enclosed or formed within existing spaces. To this end, the most recent work involves the use of an audio controlled laser system that emulates the behaviour of the Ocilloscope in that right and left audio signals control the image response on the x and y plains. There is a significant technical difference, however, between the continuous analog response of the electromagnets in an Oscilloscope and the high speed stepper motors that rotate the tiny mirrors used to deflect the laser light. The difference is essentially that between an analog system where continuous voltage is sent to electromagnets and a digital system where each step of the stepper motor is a discrete ‘bit’ of information. In practical terms this has markedly restricted the frequency range available for the audio control of the laser. As higher frequencies put the stepper motors under increasing strain the tendency is toward lower frequency stimulus that produces simplified and less textured geometries.
Image Rob Fox
The impetus behind the laser work is to create the sensation of being inside the cathode ray tube. The work becomes three dimensional with the entire space occupied by the audio generated light patterns. Although the direct relationship between what is heard and what is seen is maintained, the relationship has also developed to include seeing what you feel. To achieve this in performance requires a sound reproduction system capable of low frequency response. With the recent addition of a second laser system, future research and development will move toward pointing lasers toward each other across large spaces, allowing the two to meet at a central point. The potential for audience envelopment is greatly increased with such a system.
Recent work has also involved the construction of a one way mirrored cube (reflective surfaces facing inward). Laser light is fired onto the mirrored surfaces and manipulated with audio to create ‘standing wave’ situations, the fluidity of the sound’s geometry becoming solid through reinforcement. In this context, audio-visual ‘solids’ can be generated, which could morph over time with changes in audio signal phase and frequency. The introduction of smoke into this enclosed space will create a three dimensional visual model (akin to a hologram) that will respond directly to any change in the incoming sound source. At this stage of the development the sound source is purely electronic and self-contained, but there is the possibility of expanding this to include environmental sounds and even interactive triggers designed to engage the viewer in the temporal unfolding of the work.
All of these endeavours attempt, in some way, to create a synaesthetic experience that connects the visual and aural senses at the level of electrical signal. Of course, this work cannot induce the neurological condition of synaesthesia, but simply suggest it. The use of electrical signal to do so has some poetic connection, perhaps, with the electrical impulses that connect parts of the brain together. The powerful reaction that synaesthetic audio visual artworks can induce in non-synaesthetes may go some way to confirming the hypothesis that we are all born with the various areas of the brain connected and simply learn to separate them over time.
Watch More
Please be aware the work in these video’s contain flashing and flickering lights.
An excerpt from a Robin Fox performance at Guildford Lane Gallery Melbourne, 2008.
Robin Fox driving a jaw-dropping laser show with glitchy experimental electronic music.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Australia.