AB
AB
AiBoss

All Categories

project

FunClip - An open-source AI-powered automatic video editing tool from Alibaba DAMO Academy

FunClip is a fully open-source, locally deployed automated video editing tool developed by Alibaba DAMO Academy. It achieves automated speech recognition for videos by utilizing the FunASR Paraformer series models from Alibaba Tongyi Labs...

What is FunClip?

FunClip is a fully open-source, locally deployed automated video editing tool developed by Alibaba DAMO Academy's Tongyi Lab. It achieves automated speech recognition for videos by utilizing the FunASR Paraformer series models from Alibaba Tongyi Lab. Users can quickly trim desired video segments based on the recognition results, selecting text snippets or specific speakers. FunClip's features include integrated high-precision Chinese ASR models, support for customized hotwords, speaker recognition, and a Grado interactive interface, making video editing simple and convenient. Furthermore, it supports multi-segment editing and can automatically generate SRT subtitle files.

FunClip's main functions

  • Automated speech recognitionUsing the FunASR Paraformer series models from Alibaba Tongyi Labs, speech recognition is performed on videos to convert speech into text.
  • Hot word customizationBy using the integrated SeACo-Paraformer model, users can specify certain entity words, names, etc. as hot words to improve the recognition accuracy of specific words.
  • Speaker recognitionIt integrates the CAM++ speaker recognition model, allowing users to crop video segments from specific speakers based on automatically recognized speaker IDs.
  • Video trimmingUsers can select text segments or specify speakers from the recognition results and obtain the corresponding video segments by clicking the cropping button.
  • Gradio User InterfaceThe interactive interface implemented through Grado simplifies the video editing process, allowing users to set up services on the server side and edit videos through a browser.
  • Multi-segment editing supportFunClip allows users to edit videos into multiple segments, providing flexible editing capabilities.
  • Automatic subtitle generationThe edited video can automatically generate SRT subtitle files for the entire video and the target segment, making it convenient for users to add or edit subtitles.
  • command line callIn addition to the graphical interface, FunClip also supports video recognition and cropping via command line, providing a more flexible way of using it.

FunClip official website entrance

How to use FunClip

Method 1: Deploy the Grado version locally

First, ensure that Python and Git are installed on your computer. Then, open a terminal and enter and run the following commands in sequence.

git clone https://github.com/alibaba-damo-academy/FunClip.git

cd FunClip && pip install -r ./requirements.txt

python funclip/launch.py

Finally, the browser accessed...localhost:7860Experience FunClip now!

Method 2: Experience the Moda Community Demo Online

accessOnline Demos provided by Moda CommunityThen follow the steps below:

  1. Upload a video or audio file (or try the use case below), and click the "Recognize" button.
  2. Copy the desired text from the recognition results to the upper right corner, or set the speaker identifier on the right, and configure the offset and subtitle settings (optional).
  3. Click the "Crop" button or the "Crop and Add Subtitles" button to get the results.

Method 3: Run relevant commands via command line

Open the terminal and enter and run the following commands in sequence.

# 1. 克隆GitHub源代码
git clone https://github.com/alibaba-damo-academy/FunClip.git
# 2. 安装Python依赖
cd FunClip && pip install -r ./requirments.txt
# 3. 添加视频进行识别
python funclip/videoclipper.py --stage 1 \
--file examples/2022云栖大会_片段.mp4 \
--output_dir ./output
# ./output/目录下将输出识别结果和.srt字幕文件
# 4. 裁剪命令
python funclip/videoclipper.py --stage 2 \
--file examples/2022云栖大会_片段.mp4 \
--output_dir ./output \
--dest_text '我们把它跟乡村振兴去结合起来，利用我们的设计的能力' \
--start_ost 0 \
--end_ost 100 \
--output_file './output/res.mp4'
Back