Authors: Ju-Sung Ham(stephen93@snu.ac.kr), You Jeong Hong, Jeong Mi Park, Jonghwa Park, Juhan Nam, and Kyogu Lee
Keywords: Emotion communication; Expressive performance; Classical Piano Pieces; Musical training;
Backgrounds
Figure 1. Musical factors that affects the emotional expression. From Livingstone et al. (2010)
•
Communication of emotion through music is affected by the factors related to composer,
performer and listener.
•
For example, Quinto & Thomson (2013) revealed that listeners’ ratings on the pleasure dimension were more changeable by the composer related factors (e.g., mode, melody), while ratings on the arousal dimension were more changeable by the performer related factors (e.g., tempo, dynamics).
•
Also, Akkermans et al. (2019) found that among several individual factors of listeners, only the level of musical training (Gold-MSI) significantly impacts listeners’ probability of correctly identifying the intended emotion.
•
However, these findings are mainly derived from monophonic melodies, which the majority of prior works have employed for experiments.
Aims
•
Therefore, our study aims to further explore the impact of factors that are related to composer, performer, and listener on emotion communication through polyphonic classical piano pieces, which are frequently consumed in our daily lives.
•
To investigate the accuracy of communication in a more quantitative way, we adopted the three-dimensional model of emotions (Mehrabian, 1996), which consists of pleasure, arousal, and dominance (PAD).
Figure 2. PAD emotional space.P for pleasure, A for arousal, and D for dominance. From Gonzalez-Sanchez et al. (2017).
Recording and Listening Experiment
Figure 3. Experimental Procedure.
•
We asked 22 professional pianists to choose one or two classical pieces and perform it both in an emotion that the original piece evokes (i.e., original version) and in four modified emotions based on the dimensions divided by valence and arousal (i.e., emotion-modified version), resulting in recordings of 24 classical piano pieces.
•
The performers reported the levels of PAD expressed in each of their performances on a 9-point scale. Then, 180 participants listened to 8 out of the 24 music pieces randomly sampled with replacement.
•
For each piece participants listened to 2 pairs of performances; the original version and a randomly selected emotion-modified version. They provided the PAD ratings for each performance, and also answered the questions about their musical backgrounds.
Analysis
•
For analyses, the emotion ratings on the original version were subtracted from the paired emotion-modified versions, defined as the relative score hereafter.
•
We evaluated the accuracy of communication between performers and listeners using new measures we defined ourselves; the direction accuracy and the L-P distance (Listener-Performer distance).
•
For the direction accuracy, it is assumed as a correct communication when the sign of the relative score matches between the performer and the listener (e.g. when both of the signs are positive, negative, or zero).
•
We calculated the L-P distance by taking an absolute value of the difference between the relative score of the performer and the listener.
•
Given these measures, we performed multivariate linear regression using the accuracy measure as a dependent variable and two composer factors and listeners’ individual factors as independent variables. Therefore, the final model included the following variables: mode (major or minor) and compositional period (baroque, classicism, romanticism, modern) of the piece, listener’s familiarity and preference for the piece, listener’s music listening hours per day, duration of musical training, the importance of music in their lives, and their age and gender.
Results
Table 1. Communication accuracy for each emotional dimension.
•
As a result, the total direction accuracy for all three dimensions was 7.4%, which was slightly higher than the chance level (3.7%).
•
For each of the PAD dimensions, the accuracy was 33.0%, 49.3%, and 38.3%, respectively.
•
The mean L-P distance for each PAD dimension was 3.2, 2.5, and 3.1, respectively.
Figure 4. Effect of training duration on emotion decoding accuracy.
•
The multiple linear regression model was only significant for arousal dimension of both the direction accuracy (Adj. R2 = 0.13, p < 0.001) and the L-P distance (Adj. R2 = 0.13, p < 0.05).
•
The model revealed that the direction accuracy of arousal dimension is positively correlated with training years (p < 0.001), and the L-P distance of arousal dimension is negatively correlated with training years (p < 0.005).
Conclusion
•
Taking the findings together, we conclude that in polyphonic classical piano music, performers’ intention in the arousal dimension is communicated more accurately than those in pleasure and dominance dimensions
•
Listeners’ musical training can improve the accuracy of such communication.
•
This implies that the findings in monophonic music have the potential to be expanded to more complex music.
References
Akkermans, J., Schapiro, R., Müllensiefen, D., Jakubowski, K., Shanahan, D., Baker, D., Busch, V., Lothwesen, K., Elvers, P., Fischinger, T., Schlemmer, K., & Frieler, K. (2019). Decoding emotions in expressive music performances: A multi-lab replication and extension study. Cognition and Emotion, 33(6), 1099–1118.
Gonzalez-Sanchez, J., Baydogan, M., Chavez-Echeagaray, M. E., Atkinson, R. K., & Burleson, W. (2017). Affect Measurement: A Roadmap Through Approaches, Technologies, and Data Analysis. Emotions and Affect in Human Factors and Human-Computer Interaction.
Livingstone, S. R., Muhlberger, R., Brown, A. R., & Thompson, W. F. (2010). Changing musical emotion: A computational rule system for modifying score and performance. Computer Music Journal, 34(1), 41–64.
Mehrabian, A. (1996). Pleasure-Arousal-Dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14(4), 261–292.
Quinto, L., & Thompson, W. F. (2013). Composers and performers have different capacities to manipulate arousal and valence. Psychomusicology: Music, Mind, and Brain, 23(3), 137–150.