AB
AB
AiBoss

All Categories

project

Parakeet TDT 0.6B - NVIDIA's open-source automatic speech recognition model

Parakeet TDT 0.6B is an open-source Automatic Speech Recognition (ASR) model released by NVIDIA. It employs a FastConformer encoder and TDT decoder architecture, accelerating inference by predicting text tags and their durations, reducing computational complexity...

What is Parakeet TDT 0.6B?

Parakeet TDT 0.6B is an open-source Automatic Speech Recognition (ASR) model from NVIDIA. It employs a FastConformer encoder and TDT decoder architecture, accelerating inference and reducing computational overhead by predicting text tags and their durations. The model can transcribe 60 minutes of audio in 1 second, achieving a real-time factor (RTFx) of 3386 and an average word error rate (WER) of only 6.05%. On the LibriSpeech-clean dataset, its WER is as low as 1.69%, ranking first on the Hugging Face Open ASR Leaderboard.

Main functions of Parakeet TDT 0.6B

  • Rapid transcriptionIt can process 60 minutes of audio in 1 second, which is 50 times faster than the existing mainstream open source ASR models.
  • High-precision transcriptionOn Hugging Face's Open ASR Leaderboard, its word error rate (WER) is as low as 6.05%, ranking among the top open-source models.
  • Lyrics transcriptionIt pioneered the ability to transcribe songs into lyrics, applicable to the music and media industries.
  • Text FormattingSupports digital and timestamp formatting, improving the readability of meeting minutes, legal transcripts, and medical records.
  • Punctuation restorationIt can automatically generate punctuation marks and capitalization, making it easier to read and for further natural language processing.
  • High real-time factorRelying on NVIDIA's TensorRT and FP8 quantization technologies, its real-time rate (RTF) reaches 3386.

Technical Principles of Parakeet TDT 0.6B

  • encoderIt adopts the FastConformer architecture, which combines the global attention mechanism of Transformer with the local modeling capability of convolutional networks, and can efficiently process long speech.
  • decoderIt uses the TDT (Transducer Decoder Transformer) architecture, which combines the efficiency of traditional Transducers in streaming speech recognition with the advantages of Transformers in language understanding.
  • Overall structureThe model has an encoder-decoder structure with 600 million parameters and supports quantization and fusion kernels to improve inference efficiency.
  • Training dataThe dataset is trained on a multi-source speech corpus called Granary, which contains approximately 120,000 hours of English audio, including 10,000 hours of manually annotated data and 110,000 hours of high-quality pseudo-labeled speech.
  • Inference optimizationOptimized for NVIDIA hardware, it combines TensorRT and FP8 quantization techniques to achieve extreme acceleration, with a real-time rate (RTF) of 3386.

Parakeet TDT 0.6B project address

Application scenarios of Parakeet TDT 0.6B

  • Call CenterReal-time transcription of customer conversations generates work order summaries, improving customer service efficiency.
  • Meeting minutesAutomatically generates meeting minutes with timestamps, making it easy for participants to quickly review and organize them.
  • Legal and medical recordsAccurately transcribe legal cases and medical records, improving document readability and accuracy.
  • Subtitle generationAdd subtitles quickly to video content to enhance the viewer experience.
  • Music IndexIt transcribes song content into lyrics, making them suitable for music and media platforms, thus expanding the indexing and analysis of music content.
  • Educational TechnologyIt supports pronunciation assessment features in language learning applications, helping students learn languages better.
Back