Embodied Cognition and the Soundtrack’s Spatiotemporal Contract

Research output: Contribution to conferencePaper


Cinema’s audiovisual language is often treated in terms of distinct planes of image and sound, with the soundtrack itself comprising multiple modes ranging from established musical structures to more extensively textural sound design, much of which may serve affective framing functions rather than conforming to a simple diegetic/non–diegetic dichotomy. Some authors have sought to elucidate the mechanisms and functions of how sound and image may influence each other, most notably Chion’s (1990) audiovisual contract. Whilst treating music and sound design as separate may provide for some specificity within our analytical frameworks, the acoustic circumstances of many contemporary soundtracks provide examples in which sound texture, gesture and timbre combine to delineate spatial attributes and territories. Furthermore, commentary relating to the haptic score (Mera, 2016) has highlighted imperatives around considering music and sound design in integrated contexts (Kulezic-Wilson, 2019).

This paper will seek to develop the approach Mark S. Ward (2015:166) proposes for considering the cinematic soundtrack via a spatiotemporal contract based upon concepts from embodied cognition, most specifically, the image schema theory of Lakoff and Johnson (1999), placing particular emphasis on the potential for audio to articulate and structure perceptual space. It will seek to explore aspects of cross–modal integration between both music and sound design and sound/music and the image via spatiotemporal concepts and models derived from image schema theory, with a particular emphasis on embodied spatial models of timbre, elaborating upon the timbre–space models of Grey (1977) through the application of embodied concepts; see Roddy and Bridges (2018). In more integrated soundtracks, spatial perspective, via timbral articulations, can be seen to grow in importance beyond the simple delineation of mental/internal and physical/external action. Timbral gestures are seen to provide the communication channels facilitating cross–talk between axes of spatial (delineation/diffusion) and temporal (integration/disintegration) structures. As the soundtrack, in this context, is considered primarily in terms of the implications of its auditory materiality, the visual territories are reinforced or undercut with affective auditory planes whose syntax and structure seems to function in relation to models from embodied cognition. Cinema, as a multimodal, audiovisual ‘language’, is thus a contract between space and time via mechanisms and frames from embodiment. Embodied models of timbre within the soundtrack can therefore provide one means by which the soundtrack can be treated in an integrated manner, both in its own right, and in relation to the visual.
Original languageEnglish
Publication statusPublished (in print/issue) - 9 Nov 2019
EventSound/Image 2019: Exploring Sonic and Audio-Visual Practice - Greenwich University, London, United Kingdom
Duration: 9 Nov 201910 Nov 2019


ConferenceSound/Image 2019
Country/TerritoryUnited Kingdom


  • Sound
  • Music
  • Film
  • Spatiotemporal
  • embodied cognition
  • Plurality
  • Chernobyl
  • Revenant


Dive into the research topics of 'Embodied Cognition and the Soundtrack’s Spatiotemporal Contract'. Together they form a unique fingerprint.

Cite this