AB
AB
AiBoss

All Categories

project

TIGER - A lightweight speech separation model developed by Tsinghua University

TIGER (Time-frequency Interleaved Gain Extraction and Reconstruction Network) is a lightweight speech separation model proposed by a research team at Tsinghua University. It utilizes a time-frequency interleaved modeling strategy...

What is TIGER?

TIGER (Time-frequency Interleaved Gain Extraction and Reconstruction Network) is a lightweight speech separation model proposed by a research team at Tsinghua University. Through a time-frequency interleaved modeling strategy, combined with frequency band segmentation and a multi-scale attention mechanism, it significantly improves speech separation performance while reducing the number of parameters and computational cost. The core of TIGER lies in its innovative time-frequency interleaved modeling module (FFI), which efficiently integrates time and frequency information to better extract speech features. The model introduces a multi-scale selective attention module (MSA) and a full-frequency/frame attention module (F³A) to further optimize feature extraction capabilities.

TIGER's main functions

  • High-efficiency speech separationTIGER can efficiently separate different speakers in mixed speech through its innovative time-frequency cross-modeling module (FFI) and multi-scale attention mechanism.
  • Low computational cost and low parameter countThe model's performance is still comparable to the most advanced models currently available, even after compressing 94.3% of the parameters and 95.3% of the computation.
  • Adaptation to complex acoustic environmentsTIGER uses the EchoSet dataset to simulate noise and reverberation in real-world scenarios, improving the model's robustness in complex environments.

TIGER's technical principles

  • Time-frequency cross modeling strategyThe core of TIGER lies in its Time-Frequency Cross-Modeling (FFI) module, which effectively integrates time-frequency features by alternating the processing of time and frequency information. The module includes frequency paths and frame paths, each containing a Multi-Scale Selective Attention (MSA) module and a Full-Frequency/Frame Attention (F³A) module, which can fuse local and global information to improve speech separation performance.
  • Frequency band segmentationThe energy of speech signals is unevenly distributed across different frequency bands, with the mid-to-low frequency bands containing more speech information and the high frequency bands containing more noise and details. TIGER uses a frequency band segmentation strategy to divide the frequency band into sub-bands of different widths, reducing computational load while allowing the model to focus on key frequency bands.
  • Multiscale attention mechanismTIGER introduces a multi-scale selective attention module (MSA), which integrates local and global information through multi-scale convolutional layers and selective attention mechanisms, enhancing the model's ability to extract multi-scale features.
  • Overall processThe overall process of TIGER consists of five parts:
    • encoderThe mixed audio signal is converted into a time-frequency representation through short-time Fourier transform (STFT).
    • Frequency band segmentation moduleThe entire frequency band is divided into multiple sub-bands, and each sub-band is converted into a unified feature dimension through one-dimensional convolution.
    • SeparatorIt consists of multiple time-frequency cross-modeling modules (FFI) used to extract the acoustic features of each speaker.
    • Bandwidth recovery module: Restore the subband to the full frequency range.
    • decoderClear speech signals for each speaker are generated through inverse short-time Fourier transform (iSTFT).

TIGER's project address

Application scenarios of TIGER

  • Meeting and speech minutesIn multi-speaker meetings or presentations, TIGER can efficiently separate the voices of different speakers, improving the efficiency and accuracy of meeting recording.
  • Video editing and productionIn video content creation, TIGER can accurately separate the voice of the host from the background or other people's voices, making post-production and editing easier.
  • Movie audio processingTIGER excels in film audio separation tasks, separating vocals, music, and sound effects, enhancing the flexibility and quality of audio processing.
  • Intelligent voice assistantIn smart voice assistant applications, TIGER can help separate user speech from background noise, improving the voice interaction experience.
Back