WithFeel: embodiment as a framework for accessible music composition, performance and design in virtual reality

Abstract

In the last 30 years of Accessible Digital Music Interface research, some broad trends can be identified within the research literature. As will be demonstrated, the overwhelming majority of interfaces within the literature are tangible in nature and focussed on musical performance. Lacking in the literature are interfaces that support music composition. The introduction of Virtual Reality Head Mounted Displays into the marketplace in recent times presents a new avenue for Accessible Digital Music Interfaces. Virtual Reality presents designers with the possibility of exploring and controlling the spatial and embodied properties of virtual environments through novel interactions. These embodied interactions not only suggest new methods of performance but also of notation and composition that may overcome barriers to inclusion.

This thesis, however, goes one step further and uses the same embodied theories used for music creation within the design process itself. The thesis describes the development of software applications with three musicians with intellectual impairments and one musician with quadriplegic cerebral palsy. The methodology draws from research in design, interaction, and composition; combining them into one singular framework unified by concepts derived from embodied cognition. Firstly, designer and participants define a shared musical/design language and then, through iterative participatory design cycles explore usability, multimodal interaction, new affordances of virtual reality and new methods of composition to create a system that overcomes a hierarchy of acceptance found in the literature. The resulting output of a composition interface (Compose WithFeel), virtual reality performance interface (WithFeel VR)and associated audio plugins empowers the participants to compose, perform and conduct their own material.

Date of Award	Apr 2023
Original language	English
Supervisor	Frank Lyons (Supervisor), Brian Bridges (Supervisor) & Rob Casey (Supervisor)