Granite-4.0-1b-speech - IBM's open-source multilingual speech model

What is Granite-4.0-1b-speech?

Granite-4.0-1b-speech is an open-source, 1 billion-parameter multilingual speech model from IBM. It supports speech recognition in English, French, German, Spanish, Portuguese, and Japanese, as well as bidirectional translation with English. It also supports one-way translation from English to Italian and from English to Mandarin. The model is based on a 16-layer Conformer encoder and Q-Former projection layer architecture, achieving an average word error rate of only 5.52% on the HuggingFace Open ASR Leaderboard. It supports speculative decoding to accelerate inference and is compact enough for enterprise-level speech transcription and edge device deployment.

Main functions of Granite-4.0-1b-speech

• Multilingual speech recognitionIt supports automatic speech recognition in six languages: English, French, German, Spanish, Portuguese, and Japanese, and can convert speech input into corresponding text output.
• Two-way voice translationThe model enables bidirectional automatic speech translation between English and the six languages mentioned above, allowing users to conduct real-time speech translation and communication between different languages.
• One-way speech translationThe model supports one-way speech translation from English to Italian and from English to Mandarin.
• Keyword bias identificationThe model has the ability to suggest keywords, and users can add specific terms to the end of the suggested words to enhance the accuracy of recognizing names, place names and professional abbreviations.
• Security protection mechanismWhen receiving audio prompts in unfamiliar or abnormal formats, the model automatically reverts to the default transcription mode, effectively reducing the security risks posed by adversarial input attacks.
• High-efficiency reasoning accelerationThe model supports speculative decoding techniques and, with optimized Conformer encoder training, achieves high-speed inference with a real-time factor of 280.
• Edge device adaptationThanks to its compact architecture with only 1 billion parameters, the model can be efficiently deployed and run on resource-constrained edge devices.

Key information and usage requirements for Granite-4.0-1b-speech

• DevelopersIBM.
• Core CompetenciesIt supports recognition of six languages: English, French, German, Spanish, Portuguese, and Japanese, as well as bidirectional translation with English. It also supports translation from English to Italian and Mandarin.
• Environmental requirements: Transformers ≥ 4.52.1, torchaudio, soundfile; supports CUDA and Apple Silicon.
• Audio requirementsMono, 16kHz sampling rate, via<|audio|>The tag is introduced.
• Safety RecommendationsUsed in conjunction with Granite Guardian to detect risky content.

The core advantages and value of Granite-4.0-1b-speech

• Ultimate efficiencyWith a lightweight architecture of only 1 billion parameters, it achieves an inference speed of 280 times that of real-time factors, maintaining excellent recognition performance while significantly reducing computing resource consumption, making it particularly suitable for deployment and operation on edge devices and in resource-constrained environments.
• Accurate identificationThe model achieved an average word error rate of 5.52% on the HuggingFace Open ASR Leaderboard benchmark and an excellent performance of 1.42% on the Librispeech Clean dataset, with accuracy comparable to similar models with a larger number of parameters.
• Multilingual coverageA single model simultaneously supports speech recognition in six languages: English, French, German, Spanish, Portuguese, and Japanese, as well as bidirectional speech translation between these languages and English, meeting the multilingual processing needs of multinational corporations in their global operations.
• Enterprise securityThe model has a built-in security protection mechanism that automatically reverts to the default transcription mode when it detects unfamiliar or abnormal input prompts, effectively avoiding the risk of adversarial attacks. Combined with the Apache 2.0 open source license, it provides legal protection for enterprise commercial use.
• Flexible and easy to useThe model natively supports multiple mainstream inference frameworks such as Transformers, vLLM, and MLX, and provides a keyword list bias function, allowing users to enhance the recognition accuracy of specific terms, names, and abbreviations by customizing prompts, thus adapting to diverse business scenarios.

How to use Granite-4.0-1b-speech

• Install dependencies:implement pip install transformers torchaudio soundfile Install necessary libraries if using Apple Silicon. mlx-audio.
• Loading Model:pass AutoProcessor.from_pretrained and AutoModelForSpeechSeq2Seq.from_pretrained Load the processor and model separately, and set them up. torch_dtype=torch.bfloat16 Enable efficient reasoning.
• Prepare audioLoad a mono audio file with a 16kHz sampling rate to ensure that the audio dimensions meet the model input requirements.
• Build tips:use <|audio|> Marking to introduce audio, in conjunction with apply_chat_template Generate dialogue format prompts, and add a keyword list at the end to achieve bias recognition.
• Execution reasoningThe processor is invoked to convert cues and audio into model input, via... model.generate Generate output, decode it, and obtain the final text result.
• Deployment methodChoose vLLM for high-concurrency service deployment, or run MLX locally on Apple Silicon devices.

The project address for Granite-4.0-1b-speech

• HuggingFace model libraryhttps://huggingface.co/ibm-granite/granite-4.0-1b-speech#granite-40-1b-speech

Comparison of Granite-4.0-1b-speech with similar competing products

Dimension	Granite-4.0-1b-speech	OpenAI Whisper
Language support	Six input languages, focusing on major languages in Europe, America, and Asia.	Supports 99 languages, offering broader coverage including Chinese language recognition.
Model size	1 billion parameters, lightweight and efficient	Parameters range from tiny to large.
Special features	Keyword bias, accelerated speculative decoding	Highly versatile, multi-tasking end-to-end
Open source license	Apache 2.0, commercially friendly	The MIT license is also open source.
Applicable Scenarios	Enterprise-grade edge deployment, real-time translation	Multilingual universal recognition, research and exploration

Application scenarios of Granite-4.0-1b-speech

• Meeting minutes transcriptionThe model can convert multilingual conference audio into text records in real time, supporting participant speech recognition in six languages: English, French, German, Spanish, Portuguese, and Japanese, and automatically generating structured meeting minutes.
• Cross-border customer service supportIt supports handling multilingual customer calls, enabling real-time speech-to-text transcription and bidirectional translation with English, helping customer service personnel understand customer needs in different languages and respond accurately.
• Video subtitle generationIt automatically generates accurate subtitles for multilingual video content, and ensures accurate recognition of professional terms, names of people and places through keyword biasing, thereby improving the quality of subtitles.
• Real-time simultaneous interpretationProvides real-time voice-to-voice translation assistance in international conferences or business negotiations, supporting mutual translation between six languages and English, reducing the barrier to cross-language communication.