The Void Between Visual and Aural Perception

The starting point of this research is the fact that the representation of reality through an image is flattened and presented on a two-dimensional surface, while sound tends to move through space and therefore no dimensional loss occurs. The aim is to add an additional layer of depth to reality through sound, generated on the basis of visual elements (i.e. light and color). By doing so, the research investigates whether or not it is possible to enhance the experience of depth by translating light into sound? This study analyses the internal structure of light and sound as a wave, to find a valid model of translation. Furthermore, in order to see how our brain responds to stimuli, the phenomena of multisensory experience known as synaesthesia are examined. With the help of examples throughout history such as color organs, abstract cinema and (live) immersive installations, the connection from sound to color will be shown. We will see how neuroscience and technology have helped visually impaired persons to overcome their physical handicaps. The dossier concludes that hearing has a wider range than sight does, which is the reason why an experience of space on the basis of sound will be more intense. This conclusion introduces an experimental project dealing with transmission from one medium (visual) to another (aural), showing a kind of synthetic art. By rendering the audible from the visible (i.e. translating light input into audio output) via Pure Data software, the project's main concepts such as void, perspective and time, are exposed.

The Void Between Visual and Aural Perception
photo credit Petar Kufner


extract from The Void Between Visual and Aural Perception
(Time, perspective, void)

Light and sound are elements used, but while discussing them this project refers to image (photography) as a recorded representation of light in the case of light,[1] and depth in the case of sound. When dealing with vision, not only the image (representation of the visual) but also the human view is "framed". By "framing" it returns to Heidegger's statement mentioned in the introduction of reducing everything to an image. This framing (of reality) refers to the visual as the image to its representation. We can state that view equals representation. Additionally, by framing reality we place ourselves in a dominant position. Painters and later photographers did the same thing while trying to communicate and reflect on reality. Therefore, another element that this project deals with is the observer's point of view - “ego centered principles of single point perspective,”[2] (process similar to qualia). The idea that sound is more neutral than perspective, by being omnipresent and by giving a complete vision, is no longer satisfactory. Artists in general, but also those dealing with sound (selecting and finding the right spot, deciding on when to start, end and make a cut) could be correlated to a single point of view. Thus, working with sound includes subjectivity. Sound characteristics that reach us personally, regardless to which stimuli we are reacting, remain equally personal as a person who tries to communicate communicates subjectively (personal subjective experience, qualia). Still, we can use the characteristics of sound moving through space to add depth and to position ourselves within space. By tracing the space with the ear, as observed by Pallasmaa, we can measure it and make its scale comprehensible. "Hearing structures articulates the experience and understanding of space. We are not normally aware of the significance of hearing in spatial experience, although sound often provides the temporal continuum in which visual impressions are embedded. When the sound track is removed from a film, for instance, the scene loses its plasticity and sense of continuity and life. Silent film had to compensate for the lack of sound by a demonstrative manner of over-acting."[3] Furthermore, space, in case of an image, is flattened - presented on a two-dimensional surface, while sound tends to move through space and therefore no dimensional loss occurs. This has been observed by Michel Chion in the medium of film, “[...] if sounds are easily projected by the spectator onto the film image, it is because the image is circumscribed by a frame that can be located in space, whereas sound lacks a frame.”[4] The unity between these two domains, present in both objects and nature, could be compared to the medium of cinema, as Robert Breton observes in a poetical way: “Images and sounds, like strangers who make acquaintance on a journey and afterwards cannot separate.”[5] Cinema works with light and sound simultaneously, cinema is time. It is defined as fps (frames per second) displaying frames in quick succession, which create the illusion of motion. Each frame is a still image. If we want to record light using the medium of photography, it is possible by working in couple[6] light time. The same thing happens with sound; in order to be perceived or registered it needs a time space (medium for propagation) connection. Light, visually represented using colors and perspective, and sound, manifested through object and space, have time in common as an important parameter in the connection light to sound. Time also means movement. Movement is expressed in the change and the duration in space. "Art has become a question of movement, of what we get to rather than the abolition of this "getting to" (...). Art is only the trace of its own action".[7] According to Juahani Pallasmaa "I regard an object, but sound approaches me; the eye reaches, but the ear receives."[8] Therefore, if we treat vision - reaching, as directional and sound - receiving, as omni-directional, is it possible to perceive something merely visual as something aural by inversing reaching with receiving? The project's focus lies on the space that stands outside the frame (i.e. physical frame recalling our vision). The space that stands outside the “frame”, thus between the viewer and the observed object, is defined as void. In this case void is considered an empty space (i.e. the space between infinity and the subject) where the possible translation occurs, or rather the space where this translation takes place before it reaches us. According to Marcel Duchamp "Art is not what we see; it is in the spaces between."[9] Or as observes Anish Kapoor: "The void is not silent. I have always thought of it more and more as a transitional space, an in-between space. It's very much to do with time. I have always been interested as an artist in how one can somehow look again for that very first moment of creativity where everything is possible and nothing has actually happened. It's a space of becoming… 'something' that dwells in the presence of the work… that allows it or forces it not to be what it states it is in the first instance."[10] The void will be exposed through its presence and aura,[11] by experiencing the “authentic” with our senses. "Everything we see hides another thing, we always want to see what is hidden by what we see"[12] observes René Magritte; this will occur through time to reveal an auditory presence of the visual. Sound itself is contained within the object perceived. At the same time a light is being cast, the shadow gives depth and materiality to the object. Nonetheless, it is also a presence of something missing, not visually recognized: “it rather points toward a gap in the field of visible, toward a dimension of what eludes our gaze.”[13]

[1] "Photography has never actually been anything more than the first of these 'arts of light' that have little by little contaminated the perceptible through a 'photosensitivity' [...]." In Paul Virilio, Art as Far as the Eye Can See, (Oxford: Berg Publishers, 2007), 117.
[2] Christian Metz, cit., in Paula Carabell, "Photography, Phonography, and the Lost Object," Perspectives of New Music, 40, no. 1 (2002): 179, http://www.jstor.org/stable/833552 (accessed March 19, 2012).
[3] Juhani Pallasmaa, The Eyes of the Skin, Architecture and the Senses, (Chichester: John Wiley & Sons Ltd., 2005), 49.
[4] Michel Chion. In Patricia Kruth and Henry Stobar, Sound, (Cambridge: Cambridge University Press, 2007), 204.
[5] Michel Chion, Audio-Vision, Sound on Screen (New York: Columbia University Press, 1994), xvii.
[6] tr. it.,"coppia tempo diaframma" The term used to express the connection t-f (time-blend aperture). Even though it means exposure, for this purpose the Italian term suits better on discussed topic.
[7] Alain Badiou, "THE SYMPTOM 9," Some Remarks on Marcel Duchamp, June 10, 2008. http://www.lacan.com/symptom/?p=39 (accessed April 06, 2013).
[8] Juhani Pallasmaa, The Eyes of the Skin, Architecture and the Senses, (Chichester: John Wiley & Sons Ltd., 2005), 49.
[9] Marcel Duchamp quoted in Hugo Heyrman, "Art and Synesthesia: in search of the synesthetic experience," First International Conference on Art and Synesthesia, July25-28, 2005. http://www.doctorhugo.org/synaesthesia/art/ (accessed April 06, 2013).
[10] Anish Kapoor, quoted in Homi K. Bhabha and Pier Luigi Tazzi, "Anish Kapoor: Making Emptiness," Anish Kapoor (London: Hayward Gallery, 1998), 17.
[11] Walter Benjamin, The Work of Art in the Age of Mechanical Reproduction, (Prisme Key Pres, 2010), 14.
[12] Harry Torczyner, Magritte: Ideas and Images, (New York: Harry N. Abrams, 1977), 172.
[13] Slavoj Zizek, "I Hear you with My Eyes, or, The Invisible Master," Gaze and Voiceas Love Objects, (Durham: Duke University Press, 1996), 93.