An empirical comparison of program auralization techniques

Andreas Mikal Stefik

This thesis presents a new approach to using music for human computer interaction, layered program auralization. I use layers of musical structure to represent the state and behavior of a computer program while it is running, taking advantage of metaphorical relationships between musical structure and programming constructs. Layers overlap one another, and can intelligently collaborate to create meaningful mappings from program state or behavior to music. I describe three possible layers in this new system. A dynamically controlled tonal structure changes the harmony while a computer program is running. Program state is represented by changes in the orchestration during execution. Lyrics add semantic information that is difficult to represent with music alone. One possible application of layered program auralization is in debugging runtime behavior of computer programs. Three programs, with faults strategically added, were written to test the effectiveness of layered program auralization. The three programs created included a roulette game, a bank automatic teller machine, and an address book. An empirical study was conducted comparing the effectiveness of three groups of participants while debugging these programs. The first group of participants were given no auralizations, the second strictly musical auralizations, and the third musical auralizations with additional lyrics. Three sessions of experiments were run, the first of which without training into how the auralizations work. In the last two sessions of the experiment, participants in the music and music plus voice groups were given training in the auralizations. Results indicate that layered program auralization was effective in the music group for the bank example, but may not work for every type of computer program, using the current auralization design. Interestingly, users debugging control flow aspects of programs found more errors than other types of programs, like linked structures. In addition, in the music plus voice group, subjects were found to debug less effectively than the control group if they were given no training, although this effect was not seen with the music only group.

An empirical comparison of program auralization techniques

Files and links (1)

Abstract

Metrics

Details