Abstract
As digital music production has become mainstream, the selec- tion of appropriate virtual instruments plays a crucial role in de- termining the quality of music. To search the musical instrument samples or virtual instruments that make one’s desired sound, mu- sic producers use their ears to listen and compare each instrument sample in their collection, which is time-consuming and inefficient. In this paper, we call this task as Musical Instrument Retrieval and propose a method for retrieving desired musical instruments using reference mixture audio as a query. The proposed model consists of the Single-Instrument Encoder and the Multi-Instrument Encoder, both based on convolutional neural networks. The Single-Instrument Encoder is trained to classify the instruments used in single-track audio, and we take its penultimate layer’s activation as the instru- ment embedding. The Multi-Instrument Encoder is trained to esti- mate multiple instrument embeddings using the instrument embed- dings computed by the Single-Instrument Encoder as a set of tar- get embeddings. For more generalized training and realistic eval- uation, we also propose a new dataset called Nlakh. Experimental results showed that the Single-Instrument Encoder was able to learn the mapping from the audio signal of unseen instruments to the in- strument embedding space and the Multi-Instrument Encoder was able to extract multiple embeddings from the mixture audio and re- trieve the desired instruments successfully. The code used for the ex- periment and audio samples are available at:https://github.com/minju0821/musical_instrument_retrieval