AB
AB
AiBoss

All Categories

project

SafeEar - An AI-powered audio forgery detection framework jointly open-sourced by Zhejiang University and Tsinghua University.

SafeEar is an AI-powered audio spoofing detection framework jointly developed by Zhejiang University and Tsinghua University. It protects user privacy while detecting audio spoofing. It employs a decoupled model based on a neural audio codec to separate the acoustic information of speech from...

What is SafeEar?

SafeEar is an AI-powered audio forgery detection framework jointly developed by Zhejiang University and Tsinghua University. It protects user privacy while detecting audio forgery. Employing a decoupled model based on a neural audio codec, it separates the acoustic and semantic information of speech, using the acoustic information for detection to effectively prevent privacy leaks. SafeEar performs exceptionally well on multiple benchmark datasets, achieving an equal error rate (EER) as low as 2.02%, and is resistant to content recovery attacks. SafeEar provides multilingual support and has constructed the CVoiceFake dataset, containing 1.5 million multilingual audio data points, providing a valuable resource for speech forgery detection research.

SafeEar's main functions

  • Privacy-protected deepfake detectionBy separating the semantic and acoustic information of speech, SafeEar uses only acoustic information to detect deepfake audio, protecting the privacy of the speech content during the detection process.
  • Multilingual supportSafeEar can process and detect audio data in multiple languages, including but not limited to English, Chinese, German, French, and Italian.
  • Highly efficient counterfeit detectionTests on multiple public benchmark datasets show that SafeEar has an equal error rate (EER) as low as 2.02%, demonstrating efficient detection capabilities.
  • Anti-content recovery technologySafeEar combines real-world-based codec enhancements and anti-content recovery technology to maintain high detection accuracy even under adversarial attacks.
  • Augmented RealityBy simulating the diversity of audio channels in real-world environments, SafeEar enhances the model's ability to generalize to different communication scenarios.
  • Open source resourcesSafeEar provides open access to papers, code, and datasets, facilitating further research and application development within the research community.
  • Dataset ConstructionSafeEar has built the CVoiceFake dataset, which contains 1.5 million multilingual audio samples, providing a standardized benchmark for voice forgery detection.

SafeEar's technical principles

  • Semantic-acoustic information separationSafeEar uses a neural audio codec model to decouple semantic information (such as language content) from acoustic information (such as timbre, intonation, and rhythm) in audio. This separation ensures that the specific content of the audio is not revealed when detecting fake audio, thus protecting user privacy.
  • Acoustic Information AnalysisThe system analyzes only the acoustic features of the audio, not its semantic content. By focusing on acoustic features such as pitch, timbre, and rhythm, SafeEar can effectively identify fake audio.
  • Multilingual supportSafeEar can process audio data in multiple languages and uses semantic removal processing to ensure that specific semantic content is not exposed during the analysis process.
  • Anti-content recovery technologySafeEar combines real-world-based codec enhancements and anti-content recovery technology to resist various audio deepfake methods, ensuring high detection accuracy even under adversarial attacks.
  • Transformer classifierSafeEar employs an acoustic input-based Transformer classifier, which has shown potential in forgery detection, improving both accuracy and efficiency.

SafeEar project address

Application scenarios of SafeEar

  • Social media and public forumsOn these platforms, fake audio content can be used to mislead or deceive, and SafeEar can be used to detect and flag such fake content.
  • legal and judicial systemIn legal proceedings, the authenticity of audio evidence is crucial. SafeEar can help verify the authenticity of recordings.
  • Financial institutionsIn customer service and transaction verification, voice recognition systems are vulnerable to spoofed audio. SafeEar can improve transaction security.
  • Government and security agenciesIn the areas of national and public safety, detecting spoofed audio messages is crucial. SafeEar can help identify potential threats and disinformation.
  • Online EducationIn online courses and exams, SafeEar can be used to ensure the authenticity of audio materials and prevent academic fraud.
Back