ClipSketch AI - An open-source AI content creation tool that automatically extracts keyframes from videos.

What is ClipSketch AI?

ClipSketch AI is an open-source, AI-driven content creation tool designed for video creators, social media operators, and fan-made content enthusiasts. It can parse video links from Bilibili and Xiaohongshu, automatically extract keyframes and convert them into hand-drawn storyboards, while generating captions tailored for social media. The tool supports multi-source video import, high-definition playback, and frame-level tagging. It's easy to use and suitable for various creative scenarios, such as series content creation, short narrative videos, and product recommendations.

ClipSketch AI's main functions

• Video parsing and frame-level markingSupports parsing Bilibili and Xiaohongshu video links, accurately marking highlights to the frame level, and exporting the marked data as TXT timeline tags or ZIP image packages.
• AI drawing and copywriting generationBased on the Google Gemini big model, it can convert tagged video clips into hand-drawn style storyboards with one click and generate social media copy in three styles.
• Batch processing and optimizationIt has the ability to generate storyboards in batches and can be configured to use the Batch API, saving creation costs.
• Multi-platform compatibilityThe interface features a responsive design, supporting adaptation on PCs, iPads, and mobile devices, making it easy to use.
• Open source deploymentIt supports local open-source deployment, and users can configure their own Gemini API Key to achieve data privacy protection and function customization.

How to use ClipSketch AI

• Access Platform: Go to the online demo or the local deployment address for ClipSketch AI.
• Import videoEnter a video link from Bilibili or Xiaohongshu, and the tool will automatically parse the video.
• Frame-level tagsBy playing the video, you can use keyboard shortcuts (such as the spacebar to play/pause, the left and right arrow keys to adjust frame by frame, and the T key to mark) to mark the highlights.
• Generate storyboardAfter marking, click the generate button, and the tool will convert the video clip into a hand-drawn style storyboard.
• Generate copyIt can generate three styles of social media copy simultaneously, allowing users to choose the style that best suits their needs.
• Export dataThe marked data can be exported as TXT timeline labels or ZIP image packages for easy editing later.
• Local deploymentIf long-term use is required, you can deploy it locally based on the open-source code and configure your own Gemini API Key.

ClipSketch AI project address

• Github repositoryhttps://github.com/RanFeng/clipsketch-ai
• Online experience addresshttps://clipsketch-ai.vercel.app/

Application Scenarios of ClipSketch AI

• Short video creationQuickly extract keyframes from long videos, convert them into hand-drawn style storyboards, and then publish them to platforms such as Xiaohongshu and Weibo, saving time on storyboard design and copywriting.
• Social media operationsBatch processing of trending content from platforms such as Bilibili and Xiaohongshu, quickly generating derivative content tailored to the style of this account.
• Recommended AccountsExtract key features from product review videos, combine them with hand-drawn illustrations and accompanying text to enhance product seeding effectiveness.
• Production of narrative short videosTransform long-form video clips into hand-drawn storyboards for easier secondary creation and editing.
• Series content productionQuickly break down tutorial videos and generate structured storyboards to facilitate the planning and production of series content.
• Educational content creationAutomatically convert instructional videos into storyboards for course planning and presentation.