Qwen3 Embedding - A series of open-source text embedding models from Alibaba Tongyi

What is Qwen3 Embedding?

Qwen3 Embedding is a dedicated text representation, retrieval, and ranking model developed based on the Qwen3 base model. It inherits the advanced architecture of Qwen3, such as Grouped Query Attention and the SwiGLU activation function. Through multi-stage training, including large-scale weakly supervised pre-training, high-quality supervised fine-tuning, and model fusion, it improves performance and robustness. It supports multiple languages, covering 119 languages, with parameter sizes ranging from 0.6B to 8B, allowing for flexible selection of representation dimensions and optimization of instruction adaptation. It performs exceptionally well on tasks such as MTEB; for example, Qwen3-Embedding-8B achieves a Mean (Task) of 75.22 on MTEB (eng,v2) and 73.84 on MTEB (cmn,v1).

The main functions of Qwen3 Embedding

• Precise semantic captureIt can convert text into high-dimensional vectors, accurately capturing the semantic information of the text. Different text content will be mapped to different vector space locations, and texts with similar semantics will be closer in the vector space, providing a high-quality semantic representation foundation for subsequent text processing tasks.
• Multilingual supportIt supports multiple languages, covering 119 languages, and can effectively represent texts in different languages, meeting the needs of cross-language text processing and providing support for text analysis, information retrieval and other tasks in multilingual environments.
• High-efficiency search capabilityThis technology embeds text into a vector space and uses vector similarity calculations (such as cosine similarity) to quickly retrieve text content that is semantically similar to the target text. In large-scale text data, it can quickly locate information related to the query text, improving retrieval efficiency and accuracy, and is suitable for scenarios such as search engines and knowledge graphs.
• Flexible search methodsIt supports text-based searches as well as keyword or phrase-based searches. Users can choose the appropriate search method according to their specific needs to obtain more accurate search results.
• Semantic relevance rankingIn text ranking tasks, such as content recommendation in recommendation systems and answer ranking in question-answering systems, Qwen3 Embedding can rank texts based on their semantic relevance. It prioritizes texts that are more semantically relevant to the target text, improving the accuracy and relevance of the ranking results and providing users with more suitable text content.
• Personalized sorting optimizationBy combining users' historical behavior data and preference information, the text sorting results are personalized and optimized. This provides different users with text sorting that better matches their interests and needs, thereby improving the user experience.

The technical principle of Qwen3 Embedding

• Architecture design based on the Qwen3 basic modelThe Qwen3 Embedding series of models are built upon the Qwen3 basic model, inheriting its powerful multilingual text understanding capabilities. The models employ a dual-tower structure design, offering three parameter scales: 0.6B, 4B, and 8B, to meet the performance and efficiency requirements of different scenarios.
• Multi-stage training processThe training process of Qwen3 Embedding follows a multi-stage training paradigm:
  • Weakly supervised pre-trainingWe synthesize large-scale, high-quality, multilingual, and multi-task text relevance datasets based on the Qwen3 instruction model, and optimize them using an improved contrastive loss function.
  • Supervision and fine-tuningHigh-quality, small-scale data is selected from synthetic data for the supervised training phase to further optimize model performance.
  • Model fusionThe Spherical Linear Interpolation (SLERP) technique is used to merge multiple model checkpoints, thereby improving the robustness and generalization performance of the model.
• Text Representation TechnologyThe Qwen3 Embedding model accepts a single text segment as input and takes the last layer of the model as input. [EOS] The hidden state vector corresponding to the label is used as a semantic representation of the input text. This design enhances the model's ability to understand text semantics and supports customizable representation dimensions, allowing users to adjust the representation dimensions according to their actual needs.
• High-quality synthetic data utilizationBy filtering highly similar synthetic data pairs, high-quality data generated by the Qwen3-32B model was retained, further improving model performance.
• Hybrid Data StrategyBy combining labeled data and high-quality synthetic data, a balance is struck between task specificity and generalization ability.
• Model fusion technologyThe application of SLERP technology makes Qwen3 Embedding superior to single checkpoint models in terms of robustness and generalization ability.

Qwen3 Embedding's project address

• Github repository:https://github.com/QwenLM/Qwen3-Embedding
• HuggingFace model library:https://huggingface.co/collections/Qwen/qwen3-embedding
• arXiv technical paper:https://arxiv.org/pdf/2506.05176

Application scenarios of Qwen3 Embedding

• Intelligent Search and Information RetrievalQwen3 Embedding can be used in intelligent search systems to quickly retrieve documents or information that are semantically similar to the user's query through text vectorization technology, thereby improving the accuracy and efficiency of the search.
• Recommendation systemIn recommender systems, Qwen3 Embedding can generate high-quality text vectors by analyzing users' historical behavior and preferences, providing personalized recommendations. For example, on e-commerce platforms, it can recommend relevant products based on users' search history and purchasing behavior.
• Question and Answer SystemQwen3 Embedding can be used in question-and-answer systems to improve the accuracy and response speed of these systems by understanding user questions and retrieving relevant answers.
• EducationIn the field of education, Qwen3 Embedding can be used in intelligent education systems, such as generating solution steps for math problems and providing learning suggestions.