AB
AB
AiBoss

All Categories

project

MooER - The industry's first large-scale audio understanding model launched by MooThreads

MooER is the industry's first open-source audio understanding model developed by Moore Threads and trained on a domestically produced, fully functional GPU. It not only performs Chinese and English speech recognition but also has Chinese-to-English speech translation capabilities.

What is MooER?

MooER is the industry's first open-source audio understanding model trained on a domestically developed, fully functional GPU, launched by Moore Threads. It not only performs Chinese and English speech recognition but also Chinese-to-English speech translation. MooER achieved a BLEU score of 25.2 on the Covest2 Chinese-to-English test set, approaching industrial-grade performance. The Moore Threads AI team has open-sourced the inference code and 5,000 hours of training models, and plans to open-source the training code and 80,000 hours of training models to drive the development of AI speech technology.

MooER's main functions

  • Speech recognitionSupports speech-to-text conversion for both Chinese and English.
  • Voice translationIt has the ability to translate Chinese speech into English text.
  • High-efficiency trainingOn Moore Threads' intelligent computing platform, training on large amounts of data can be completed quickly.
  • Open source modelThe inference code and part of the training model have been open-sourced to facilitate community use and further research.

MooER's technical principles

  • Deep learning architectureMooER uses deep learning technology, especially neural networks, to process and understand speech signals.
  • End-to-end trainingThe model outputs text directly from the raw speech signal, eliminating the need for multiple independent modules found in traditional speech recognition systems.
  • Encoder-Adapter-Decoder Structure:
    • Encoder: Responsible for converting the input speech signal into a series of high-level feature representations.
    • AdapterUsed to adjust and optimize the model's adaptability to specific tasks and improve the model's generalization ability.
    • Decoder(Large Language Model, LLM): The final text output is generated based on these features.
  • LoRA technology: Using LoRA (Low-Rank Adaptation) technology, a parameter-efficient model fine-tuning method, improves training efficiency and effectiveness by updating only a small portion of the model's parameters.
  • Pseudo-label trainingThe pseudo-label technique is used during training, which uses the model's own predictions as training data to enhance the model's learning ability.
  • Multilingual supportMooER supports speech recognition in Chinese and English, as well as speech translation from Chinese to English, demonstrating its multilingual processing capabilities.

MooER's project address

How to use MooER

  • Get the modelAccessibleGithub repositoryObtain the code and pre-trained weights of the MooER model.
  • Environment configurationEnsure that the necessary dependency libraries and tools are installed in the computing environment, such as Python, deep learning frameworks (such as TensorFlow or PyTorch), audio processing libraries, etc.
  • Data preparationPrepare the audio data and (if necessary) the corresponding text transcription. Ensure the data format matches the model input requirements.
  • Model loadingLoad the pre-trained MooER model into the computing environment.
  • Data processingPreprocessing of audio data, such as normalization and frame segmentation, is performed to match the input requirements of the model.
  • Model InferenceThe MooER model is used to infer from preprocessed audio data to obtain speech recognition or translation results.

Application scenarios of MooER

  • Real-time speech transcriptionIn meetings, lectures, classrooms, and other settings, MooER can convert speech to text in real time, making it easy to record and review.
  • Multilingual translationIt supports voice translation between Chinese and English, making it suitable for scenarios such as cross-border meetings and international exchanges.
  • Intelligent Customer ServiceIn the field of customer service, MooER can improve customer service response efficiency and service quality through voice recognition and translation functions.
  • voice assistantIt can be integrated into devices such as smartphones and smart speakers to provide voice interaction services.
  • Educational SupportIn language learning, MooER can help learners with pronunciation correction and language translation.
Back