Associate Professor of Psychology and author of Psychology of Music: From Sound to Significance, Siu-Lan Tan explains how music affects how we perceive characters and scenes in film. This is a special contribution to the program The Art of the Score: The Mind, Music, and Moving Images, a co-presentation by World Science Festival and the New York Philharmonic.
Reflecting on his futuristic 2002 film Minority Report, Steven Spielberg said “one of the most exciting scenes” he had to shoot was this action scene in which two characters (John and Agatha) traverse a busy shopping mall with armed police in pursuit, relying on Agatha’s ability to see into the future in order to hide and successfully evade capture.
As the two figures enter the shopping mall, an instrumental rendition of Henry Mancini’s ballad “Moon River” begins. The song can be heard faintly, playing over (what we may surmise to be) loudspeakers inside the mall. Over the distant music, we hear Agatha’s urgent instructions to John as they make their way through the mall, and the heavy footsteps of a pack of uniformed police clamoring after them. The leisurely, flowing “Moon River” serves as a serene musical canvas to the winding tension and disjointed action of this scene.
But what if “Moon River” were not playing within the fictional world of the characters, where John and Agatha reside, but as a dramatic score accompanying the film scene? How would the same piece of music work as dramatic scoring and would it change our impression or reading of the scene?
The fictional world proposed by the narrative is often referred to as ‘the diegesis’ (though see Endnotes for more nuances). Diegetic music is “music produced within the implied world of the film” (Kassabian, 2001) such as the music supposedly piped into the mall where John and Agatha are on the run. Nondiegetic music, in turn, refers to music external to this ‘narrative universe’, such as music mirroring the mood and action of a car chase. What if “Moon River” were presented nondiegetically—as a dramatic score accompanying the scene, rather than playing inside the fictional world John and Agatha inhabit? How might diegetic versus nondiegetic presentation of the same piece of music shape our impressions of a film scene?
Surprisingly, a search of film music research didn’t uncover any studies along these lines. Indeed, almost all film music studies have focused solely on the effects of nondiegetic music (Tan, Spackman, & Bezdek, 2007). Diegetic music has been largely neglected in research studies, so my colleagues Matthew Spackman at Brigham Young University and Elizabeth Wakefield at Kalamazoo College and I set out to investigate!
We selected the 85-second shopping mall sequence from Spielberg’s Minority Report described above, found at 1:35:33-1:36:57 on the 2002 DVD. We prepared three versions (which we cannot post due to copyright protection):
•The Original Diegetic version was the unaltered Spielberg version with the Henry Mancini “Moon River” ballad sounding as if it were playing over speakers inside the shopping mall.
•The Nondiegetic “Moon River” version was one we created - by purchasing the same Henry Mancini “Moon River” as a single, and mixing it louder and clearer to sound like a dramatic score. Speech and sound effects from the original were retained, still crisp and audible.
•Nondiegetic ‘chase music’: As the “Moon River” music is incongruent with the mood of an action sequence, we created another nondiegetic version with John Williams’ music from Spielberg’s Empire of the Sun, which was scored for a chase sequence at 0:39:49 on the 2001 DVD. The music was taken from the CD soundtrack (Track 10, 2:03-2:30 and 2:42-3:42). Again, speech and sound effects were retained.
A total of 245 college students participated in our study. We told them that the aim of our study was “to examine how people understand story lines of film scenes”. We didn’t hint that the focus was on film music, as it was important not to direct their attention to the music. Each participant watched only one version of the film excerpt, and answered questions - including 10 scales about the scene and film characters. They indicated their responses by drawing an ‘x’ along a horizontal line. (Later, during coding, we gave each response a numerical value by measuring from the left end of the line to the center of the ‘x’ in millimeters).
When we analyzed the responses of the 111 participants who reported that they had not seen Minority Report before this study, we found that their impressions differed dramatically depending on which of the three versions they had seen. In effect, each of the three soundtracks created its own narrative, coloring the perceptions of the audience with respect to some of the most important aspects of scene and storyline.
Most strikingly, the greatest contrasts were found for responses to the diegetic versus nondiegetic version of the same piece of music: the “Moon River” ballad sounding as if inside the shopping mall (depicted by the solid red line) and the same song as an accompanying dramatic score (dashed red line).
Participants who watched the original diegetic version (created by Spielberg) rated the scene as significantly more tense and suspenseful, perceived the relationship between the two characters to be more antagonistic and hostile and less romantic, and assumed more negative intentions of the characters toward each other than those who had watched either of the two nondiegetic versions (especially nondiegetic “Moon River”). To a less dramatic degree, the fast-paced and more dissonant nondiegetic ‘chase music’ and more melodious, flowing nondiegetic “Moon River” ballad also led to somewhat different impressions of the scene and characters. However, the differences among responses were largely a function of the diegetic/nondiegetic conditions (and not ‘chase music’ versus romantic ballad, as one might expect).
To the best of our knowledge, our 2008 paper was the first published study that addressed the question in an empirical fashion. (The results reported here are only a small part of the more extensive study). As is true of any single experiment, our study has limitations and leaves questions warranting further research. In particular, the loudness of the music varies with diegetic and nondiegetic conditions in this scene in Minority Report —as the faint, distant-sounding quality of the music serves as the main cue that the music is supposedly playing over the speaker system inside the mall. These variables must be teased apart in a future study.
For instance, the astonishing “Danny Boy” sequence in Joel and Ethan Coen’s Miller’s Crossing (as briefly shown at the The Art of the Score: The Mind, Music, and Moving Images) provides a contrasting example in which (initially) diegetic music is played very loudly. Unlike Minority Report, in which the source of the music is only implied, the source in Miller’s Crossing (a gramophone player) is not only prominently shown but the stirring vocal music itself (“O Danny Boy”, a traditional Irish song) is selected by a character. In essence, the diegetic music does not simply reside in the fictional world but the musical score has been ‘selected’ by one of its inhabitants. Showing Albert Finney’s character listening to the record while lying in his bed and directing our attention to the source of the music brings the diegetic music into the conscious foreground as opposed to the sensory background of the crowded shopping mall in Minority Report. Interestingly, later as Finney’s character in Miller’s Crossing jumps out of the window to escape an assassination attempt, the lush music seems to follow him down the street at full force, without fading—a seamless transition from diegetic to nondiegetic.
While the Minority Report scene employed Henry Mancini’s prerecorded music, “Danny Boy” was recorded while performing to the film scene for Miller’s Crossing. Composer Carter Burwell recounts: Singer “Frank [Patterson] would watch the film with us, and Joel and Ethan would say, ‘Now if you could hit the word ‘bend’ here and hold it till the car explodes…’” (Morgan, 2000). Both selections of music may be described as somewhat incongruent with the mood of an action sequence and yet do not soften it - pointing to what film theorist Claudia Gorbman has referred to as “the special expressive effect of diegetic music … to create irony in a more ‘natural’ way than nondiegetic music” (1987, p.23). Both “Moon River” and “Danny Boy” have long histories and carry rich associations, evoking schema from other stories, films, and cultural and autobiographical memories. Through the connotative power of music, a scene may echo with multiple narratives.
While the expressive qualities of the music seem to be juxtaposed with the action, both songs are anchored to their scenes through their lyrics. The lyrics to “Danny Boy” linger on the theme of death. The “Moon River” lyrics (though not included in the looped instrumental version) resonate with the situations on the screen (such as “two drifters” and “wherever you’re going / I’m going your way”) as Agatha steers the pair’s flight through the mall. The lyric “rainbow’s end / waiting round the bend” coincides with the appearance of a bouquet of colored balloons coming around the corner, to resolve the scene.
It is not our intention to draw any grand conclusions about the definitive and predictable effects of diegetic versus nondiegetic music. Music interacts in dynamic and probably irreproducible ways within the complex ecology of any film scene. The specific effects of migrating a piece of music from diegetic to nondiegetic depend on the unique interplay of music and moving images. (See Cohen, 2013, for an elaborate model for music in multimedia).
However, our study suggests that the diegetic/nondiegetic distinction is perceptually salient to a general film audience—and in some cases, may lead to dramatically different perceptions of the tension of a scene, the attitudes and motives and relationships of characters, and other judgments fundamental to one’s understanding of the unfolding film narrative. Further, we have demonstrated a case in which diegetic versus nondiegetic presentation of the same piece of music produced more powerful effects than switching to a different musical soundtrack of contrasting character.
Looking to the horizon, we hope to inspire more discourse and inquiry on diegetic film music—a topic largely neglected by researchers, who have strongly privileged nondiegetic music for analysis. What do you think are the most powerful moments of diegetic music in film? What makes them particularly compelling and memorable? And what questions would you encourage psychologists, neuroscientists, and other researchers to pursue on diegetic film music?
A version of this post also appears on the Oxford University Press Blog.