Research Overview
The purpose of this project is to develop an interactive/smart music education system especially when learning instrument. For the purpose of this project, we proposed a real-time musical performance feedback system which is combination of common MIR tasks such as onset detection, Pitch detection and Score following.
This system detects pitch and onset from live audio stream and make note sequence with previous detected note events. The system aligns two note sequence which is from live audio stream and score to calculate score position, wrong note. Figure 1. shows the overview of the system. An example of rendered music note is shown in Figure 2. The music notation wass rendered using VexFlow with HTML5. The information from the backend, such as the incorrect notes and current position, are sent to VexFlow to be rendered accordingly. This HTML page is displayed on the iOS through the UIWebView class, which is used to embed web content in applications. Any additional buttons used for user interactions are coded natively through Xcode.
Figure 1.
Figure 2.
Publications
•
S. Kwon, B. Han, Y. Han and K. Lee, “A Musical Performance Feedback System Based on Real-time Score-Following”, HCI Korea 2013, 2013.
•
Y. Han, S. Kwon, K. Lee and K. Lee, “A Real-time Musical Performance Feedback System for Beginner Musician”, NIME 2013, 2013.
Project Members
Kibeom Lee, Yoonchang Han, Sungkyun Jang, Sejun Kwon
Funding Agency
Seoul Business Agency