AB
AB
AiBoss

All Categories

project

WorldCupVoice - An open-source AI-powered real-time sports commentary system

WorldCupVoice is an open-source AI-powered real-time sports commentary system. By connecting to Agora RTC live streams, it uses a visual model to analyze the game footage and generate real-time audio commentary, which is then transmitted back to the live stream room for viewers to listen to simultaneously.

What is WorldCupVoice?

WorldCupVoice is an open-source AI-powered real-time sports commentary system. By connecting to Agora RTC live streams, its visual model analyzes the game footage and generates real-time audio commentary, which is then transmitted back to the live stream studio for viewers to listen to simultaneously. The system supports multiple speech engines, including OpenAI TTS, ElevenLabs, and Fish Audio, and can be configured with commentary styles in multiple languages, such as English, French, and Chinese. Designed specifically for sporting events like the World Cup, the system also caters to the accessibility needs of visually impaired users, providing information often overlooked in traditional commentary, such as details about the stadium layout.

Main functions of WorldCupVoice

  • Real-time AI commentaryThe system samples video frames from the Agora RTC live stream, analyzes the match footage using a visual model, generates real-time audio commentary, and transmits it back to the live stream room.
  • Multi-voice engine supportCompatible with OpenAI TTS, ElevenLabs, and Fish Audio.
  • Multilingual commentarySupports multiple languages including English, French, and Chinese, and can be configured with different narration styles for different countries.
  • Live stream accessStreaming via RTMP → Agora Media Gateway → RTC channel, compatible with multiple sources including OBS, local video, and cloud streaming.
  • Browser viewing interfaceA front-end based on Next.js that synchronously plays live video, AI-generated audio narration, real-time subtitles, and monitors the narrator's status.
  • Cost control mechanismExplicit Start/Stop AI control, audience heartbeat detection, and hard session timeout prevent unlimited consumption of AI resources.
  • Visually impaired assistance modeIt provides stadium-level spatial details, making up for information that traditional human commentators often overlook.

Follow us on WeChat and reply with "open source",join inAI open source project discussion group

How to use WorldCupVoice

  • Environmental preparationConfigure Agora App ID/Certificate, OpenAI API Key, and generate backend keys.
  • Frontend startup:pnpm install → Configuration .env.local → pnpm dev。
  • Backend startup:cd server → Create a Python virtual environment → pip install → Configure TTS provider → uvicorn app.main:app。
  • Streaming settingsGenerate an RTMP streaming key via the Agora Media Gateway REST API and use OBS or local ffmpeg to stream.
  • Start of explanationEnter the live stream in your browser and click "Start AI" to activate the AI commentator.

WorldCupVoice's core advantages

  • Real-timeAI and viewers watch the same RTC stream, and the commentary delay is controllable.
  • Accessibility designProvides visually impaired users with spatial details that traditional explanations cannot cover.
  • Multiple voice stylesSupports professional sports commentary style (ElevenLabs), Chinese meme commentary (Fish Audio), and tactical analysis style.
  • Cost controllableSession-level start/stop control + heartbeat mechanism + hard timeout to avoid wasting AI resources.
  • Open source and scalableThe MIT license allows for the injection of custom competition data into the AI context.

WorldCupVoice project address

  • GitHub repositoryhttps://github.com/zicojiao/worldcupvoice

Comparison of WorldCupVoice with similar products

Comparison Dimensions WorldCupVoice WSC Sports
Product Positioning Open source real-time AI sports commentary system Enterprise-grade AI-powered sports content automation platform
Core Functions Watch RTC live stream in real time and generate AI voice commentary. Automatically identify exciting moments from the game and generate highlights, short videos, and social media content.
Real-time Live commentary, allowing viewers to listen simultaneously. It analyzes the live stream in real time, but is mainly used for automatically generating content after/during the competition.
open source It is open source under the MIT license and can be self-hosted. Closed-source SaaS platform, enterprise subscription model
Deployment method Frontend: Vercel + Backend: Railway (self-hosted) Cloud-based SaaS, API integration
Voice capabilities Supports multiple engines including OpenAI TTS, ElevenLabs, and Fish Audio. Built-in AI voiceover and multilingual support
Accessibility design Designed specifically for visually impaired users, providing detailed information about the court space. Generates general sports content, not specifically for the visually impaired.

Application scenarios of WorldCupVoice

  • World Cup/Football Match Live StreamingAdd an AI commentary channel to the live streaming platform, offering multiple languages and styles to choose from.
  • Visually impaired sports viewingProvides detailed explanations of the court space for blind or visually impaired users.
  • Niche esports live streaming: Provide automated commentary for lower-level leagues and campus matches that lack professional commentary resources.
  • Multilingual International CompetitionThe same game stream can simultaneously output AI commentary in multiple languages, including Chinese, English, and French.
  • esports commentary: Real-time AI tactical commentary for esports competitions such as LOL and CS2.
Back