Custom Music Composition with AI Systems
The overall pipeline of generating custom music that reflects the user’s taste.
•
The "AI Producer: Developing technology of custom music composition" project is a multi-faceted initiative aimed at creating personalized multi-track music compositions based on a user's musical taste. This is achieved by using a reference song and keyword-based queries for lyric generation, leveraging artificial intelligence to create unique musical pieces.
•
The project involves multiple modules, each designed to handle a different aspect of the music generation process. These include musical instrument retrieval, which identifies and extracts desired musical instruments from a reference audio file; digital audio effects modeling, which applies audio effects and performs automatic multi-track mixing and mastering; and keyword-based lyrics generation, which creates song lyrics based on keywords provided by the user.
•
While each module operates independently, they all work in synergy to produce a complete multi-track music piece. Each of these modules relies on the reference music recording provided by the user, serving to customize the final output according to the user's taste and preferences. This combination of modules, each with its unique function, makes the project a comprehensive solution for custom music composition.
Musical Instrument Retrieval
•
Musical instrument retrieval is the task of searching and identifying specific musical instruments within mixed music recordings of a collection of different musical instruments, aiming to assist music producers in selecting appropriate virtual instruments for their projects.
•
This process addresses the time-consuming and inefficient nature of manually listening to and comparing each instrument sample to find the desired sound.
•
Performing instrument retrieval not only helps in locating the desired musical instrument but also provides valuable information for instrument recognition, enabling the identification of instruments present in a given audio mix.
related publications from MARG on musical instrument retrieval
Digital Audio Effects Modeling
•
Digital audio effects modeling involves the application of artificial intelligence to analyze and replicate the audio effects applied in a given reference music piece. This process aims to recreate the unique sound characteristics that contribute to the overall sound of the reference track.
•
The task requires a deep understanding of the audio signal and the impact of different audio effects on it. The model must accurately mimic these effects and apply them both at the individual track level and during the multi-track mixing and mastering process.
•
This process is crucial in producing a final composition that closely aligns with the user's reference track in terms of sonic characteristics. By successfully replicating the audio effects of the reference, the system can create a piece of music that not only matches the user's taste but also possesses professional-level sound quality due to the effective implementation of mixing and mastering techniques.
•
related publications from MARG on digital audio effects modeling
Keyword-based Lyrics Generation
•
Keyword-based lyrics generation is a process that aims to create meaningful and coherent song lyrics based on specific keywords provided by the user, ensuring that the generated content aligns with the desired theme or message of the music piece.
•
This task requires not only the creative incorporation of the given keywords but also careful consideration of the structure, rhyme, and rhythm of the lyrics to ensure they fit naturally within the generated music composition.
•
To achieve a seamless integration with the music, the lyrics generation process must also account for the number of syllables generated by the music generation system, ensuring that the lyrics align with the melody and rhythm, ultimately contributing to a harmonious and well-structured final composition.
Contact
•
•
•