project
Audio Decomposition - an open-source music decomposition tool that converts music to sheet music with a single click.
Audio Decomposition is an audio processing technology that uses Fourier transform and envelope matching to separate individual notes and instruments in music, converting music into sheet music. The Audio Decomposition open-source project is developed by Matthew...
What is Audio Decomposition?
Audio Decomposition is an audio processing technology that uses Fourier transform and envelope matching to separate individual notes and instruments in music, converting music into sheet music. The Audio Decomposition open-source project, launched by Matthew Bird, automatically identifies and separates different instrument sounds in music without requiring external instrument separation libraries, helping users better understand and reproduce sheet music.
The main functions of Audio Decomposition
- Audio source separationSeparating the sounds of different instruments from a mixed music.
- Music to Sheet MusicConvert music files into sheet music to help users identify pitch and chords.
- Audio analysis: Analyze notes and instruments in music to provide a deeper understanding of musical structure.
- open source toolsAs an open-source project, it allows users to freely use and modify the code to adapt to different needs.
The technical principle of Audio Decomposition
- Fourier transform:The program performs a Fourier transform on the music file every 0.1 seconds to generate a spectrum.By summing the Fourier transforms of each instrument, the spectrum of the music is recreated.
- Envelope analysis:The sound wave is divided into several parts, and the maximum value of each part is taken to obtain the envelope.To further optimize the results, points with envelopes lower than the original sound waves were found, and new points were added to define the envelope.
- Disassembly of the envelope:The wave envelope is decomposed into the initial sound (attack), the sustained sound (duration), and the release sound (release).
- Waveform classification:Consider the static decay and dynamic changes of the instrument, and whether it has a phonation function.
- Bandpass filtering:Bandpass filtering is applied to the signal of each note frequency to separate the sound of a specific instrument.
- Cross-correlation and mean squared error (MSE):Find the beginning and end of each note by using the correlation between the onset and release of the instrument.Calculate the MSE of the instrument wave and the filtered audio to obtain the cost of the instrument.
Audio Decomposition project address
- Project official website:matthew-bird.com/blogs/Audio-Decomposition
- GitHub repository:https://github.com/mbird1258/Audio-Decomposition
Applications of Audio Decomposition
- Music EducationSeparating the sounds of different instruments helps students understand harmony and orchestration in musical works more intuitively.
- Music ProductionIt supports producers in extracting individual tracks from complex music for use in mixing or creating new musical elements.
- Music score transcriptionIt automatically converts audio recordings into sheet music, saving time and costs associated with manual transcription, and is especially suitable for classical music works.
- Music AnalysisIt provides in-depth analysis of the composition of musical works, offering research tools for music theorists and composers.
- Audio editingIn film and video production, it helps editors precisely process and adjust audio elements such as dialogue, music, and sound effects.