How to use AI to cover any song for free: A must-read for beginners.

With just one minute of voice message, you can...AIImitate my voice,One-clickCan you believe I can cover any song?

Last week's tutorial on creating little monk quotes received a lot of positive feedback from friends.AIThey were very interested in the cloned voices and even sent a video of a large group singing together:

useAIThe cloned voices sound like Si Lang and Zhuge Liang singing a duet online, and it's surprisingly seamless. It's quite interesting to watch, no wonder it's remained popular on the internet.

Many people are curious about how these cover videos are made.

Actually, all you need is...AICloning voices for cover songs and lip-syncing. There are also relatively mature SVC (voice-to-video conversion) technologies on GitHub, but local deployment requires high computer configuration and a large amount of voice data for training, which I find very troublesome.

Today I'm sharing two super...SimpleThis tool lets you upload a one-minute audio clip, and it's all done with just a few clicks. Even complete beginners can use it.fastGenerate cover songs.

If your computer has low specifications, just use the web version.There are a large number of voice models available in the cloud for cover songs. No training time is required, and the results are excellent. Most importantly, it's completely...free!It smells amazing when you use it.

01

Weights generation in three stepsCover Song

Select a model, upload audio, and fine-tune the settings.

Go to the Weights website (it's recommended to turn on webpage translation), the first section is...A huge number of pre-trained speech models, directlyfreeuse!Click to view all, and we can see everything from virtual characters to celebrities, such as SpongeBob SquarePants, Michael Jackson, Taylor Swift, Hatsune Miku... It's truly dazzling to choose from all of them.

I tried searching for my idol, and to my surprise, there were several voice models for him. I chose the one that ranked highest, clicked "Create" in the upper right corner, and started a new cover song task.

Click Next.

For cover audio recordings, you can either copy and upload the YouTube link directly, or drag and drop the song file directly.

It not only supports song covers, but also allows you to input text to convert to speech, or directly input your voice using your microphone to sing a cover song—it's incredibly comprehensive!

Harmony and reverb can affect the quality of a cover song, so I've chosen an audio clip of a solo a cappella performance. Let's listen to it together:

(Foxtail grass on the earthen slope)

After uploading the audio, click Next.

I uploaded an a cappella version with only vocals, and premixing was enabled. Regarding pitch, if the male voice model is covering a female song, you can lower the pitch slightly, and vice versa; here, we'll use the default setting.

Advanced options allow for more detailed adjustments. It's recommended to use the default settings first, and then fine-tune if you're not satisfied with the generated results. Click "Create" in the bottom right corner.

Go to "My Creations" in the top left corner, and you'll see cover songs being generated in a queue. It usually only takes a few minutes.

Once a success message appears in the list, you can listen to the audio and download it.

The audio files in the download page are, in order: the cover version with vocals and accompaniment (if any), the cover version with the raw vocals, the original audio, the raw vocals from the original audio, and the accompaniment from the original audio.

Let's listen to the audio of the cover version together.

🎵JJ Lin - Foxtail Grass on the Slope

The voice is very distinctive, the singing is smooth and natural, with a good sense of rhythm, and the transitions, emphasis, and high notes are all performed quite well.

The whole process takes only a few minutes and is so simple that anyone can do it.

Using the same method, I also covered Jay Chou and G.E.M.'s songs. Let's listen to them together.

(🎵Jay Chou - Foxtail Grass on the Slope)

🎵G.E.M. - Foxtail Grass on the Slope

We cut the audio into several segments according to the rhythm of the chorus, paired each segment with a Q-version portrait, uploaded them to Jimeng, and generated...Lip-syncing video.

When using images to lip-sync, be sure to select the "vivid" effect; otherwise, if only the mouth moves without the head moving, it will look very stiff.

The generated clips are then combined using CapCut (add accompaniment if available).AIThe chorus video is now complete. Let's check out the results.

How do I create my own voice model for cover songs?

02

ReplayOne-clickSeparate human voice

Training your own speech model

Let's try making a Lazy Sheep voice model and a cover song together!

First, you need to go to the Replay website and download it.up to dateThe software.

Please note that after installing the software, two prompt boxes will pop up the first time you open it. Don't rush to click on them! First, go to App-Show Settings in the upper left corner and change the file save location.

The first folder is where the exported audio will be located. The second folder contains application data, such as speech models and audio generation data. In short, don't put it on the C drive; the rest is up to you.

First, we'll extract a video of Lazy Sheep singing from Bilibili, perform pre-processing using CapCut to keep only Lazy Sheep's voice, and export it as an mp3/wav file. Then, we'll upload the audio (as shown in the image).

Select dry audio only.

For the rendering device, if your computer has integrated graphics, select CUDA; it will generate the rendering slightly faster. If your computer doesn't have integrated graphics, you'll need to switch to CPU. Leave the other settings at their defaults and click "Generate."

The generated audio will appear in the list on the left. Clicking on it will show the separated dry audio and accompaniment audio, which can be listened to and downloaded separately.

We download the extracted dry audio and upload the files to the training model section. Multiple files can be uploaded here; the more diverse the vocal range, the better the cover performance will be.

A voice model that hasn't been trained for high notes will easily distort or even crack when singing high notes, hahaha!

For the settings below, keep everything else at its default settings except for the rendering device. Then click "Create Model." This process takes a while, likely several hours.

The trained model will be displayed here.

Let's try using the Lazy Sheep model we just created to do a cover song. Upload the song you want to cover.

(🎵If the moon hasn't come yet - excerpt)

Click to select Lazy Sheep's voice model.

Leave the settings below at their default settings; they're similar to those for online cover songs. You can adjust the pitch of the vocals and instruments as needed, then click "Generate."

The cover song was finished in less than a minute. We saved both the vocal and instrumental versions separately.

Let's listen together:

🎵Lazy and leisurely - foxtail grass on the earthen slope

Download the speech model from Weights, extract it, and place it in the models folder under the application directory folder (the second location set at the beginning of the software). Then you can use it directly in Replay.

To make a choral video, simply use different models to sing the same audio clip, and then stitch them together using CapCut. Isn't that amazing?SimpleConvenient. Come listen to Lazy Sheep and Crayon Shin-chan's duet.

Using only one minute of audio material, it can achieve a decent cover song effect. Aside from the slightly slow model generation (which is my weakness...), the overall experience is very good. The vocal separation feature, in particular, is something that usually requires a membership on other platforms, but Replay and Weights both offer it.freeIt can be used an unlimited number of times, and the effect is amazing. It's hard not to love it!

Cover songs with harmonies or reverb can affect the overall effect. If we want to create a chorus, we can first create separate solo versions of the cover songs and then combine them in CapCut.

Training a model with your own voice allows you toAICover SongVariousPopularNo more worrying about singing off-key! I can master rap, pop music, and more in no time. I've learned those catchy but difficult songs in a different way.

AIWhile covers can't perfectly replicate a singer's voice and technique, you can already hear about 70-80% of the original singer's timbre. The thought of my cyber idol singing for me every day is actually quite exciting!

All tools mentioned in this article:

Weights:

Replay: