pplx-embed - A series of text embedding models launched by Perplexity

What is pplx-embed?

pplx-embed is a series of text embedding models from Perplexity, including pplx-embed-v1 for standard retrieval and pplx-embed-context-v1 for context awareness, both available in 0.6B and 4B sizes. The model transforms a causal decoder into a bidirectional encoder through diffuse continuous pre-training, achieving omnidirectional attention understanding. It natively supports INT8 and binary quantization outputs, with storage compression up to 32x. The model achieves state-of-the-art performance on benchmarks such as MTEB and ConTEB without instruction prefixes, with the 4B context model setting a new ConTEB record with a score of 81.96%.

The main functions of pplx-embed

• Dense text retrievalIt maps queries and documents to a shared semantic space, enabling efficient retrieval through near nearest neighbor search.
• Context-aware embeddingIt generates embeddings for each paragraph in a document that take into account the full text context, thus addressing the problem of insufficient semantic understanding of isolated paragraphs.
• Multilingual supportThe model covers cross-language retrieval in 30 languages, meeting the needs of global application scenarios.
• High-efficiency storage compressionNatively generated INT8 and Binary precision embeddings reduce storage requirements by 4 times and 32 times respectively compared to FP32.
• Real-time low-latency inferenceThe 0.6B lightweight version is optimized for high-throughput scenarios, balancing speed and accuracy.

The technical principle of pplx-embed

• Diffusion-based continuous pre-trainingBased on the Qwen3 base model, causal attention masking is disabled, and diffusion denoising is used to train the model to reconstruct the token of the random mask, forcing the model to use bidirectional context for understanding, and converting the autoregressive decoder into a bidirectional encoder.
• Quantitative perception trainingDuring the contrastive learning phase, INT8 precision is used throughout. Differentiable quantization is achieved through tanh mean pooling combined with pass-through gradient estimation, enabling the model to learn low-precision friendly representations natively and avoiding performance loss caused by post-compression.
• Multi-stage comparative learning courseThe process involves sequentially performing paired training to establish basic semantic alignment, contextual training to integrate document-level information, training with hard negative triples to optimize the decision boundary, and merging checkpoints through spherical linear interpolation to form the final model.

pplx-embed project address

• Project official website: https://research.perplexity.ai/articles/pplx-embed-state-of-the-art-embedding-models-for-web-scale-retrieval
• HuggingFace model libraryhttps://huggingface.co/collections/perplexity-ai/pplx-embed
• arXiv technical paperhttps://arxiv.org/pdf/2602.11151

Application scenarios of pplx-embed

• Search engines and question answering systemsAs the first-stage retrieval tool for a database of billions of web pages, it quickly recalls candidate documents for downstream reordering and generation models, supporting Perplexity's own real-time search and question-answering service.
• RAG knowledge base constructionIt generates compressed embeddings for enterprise internal documents and knowledge bases, achieving efficient semantic retrieval with extremely low storage costs. The 4B model outperforms similar high-parameter models in the BERGEN end-to-end RAG benchmark.
• Multilingual content platformThe model is applicable to scenarios such as global content recommendation, multilingual customer service systems, and multinational corporate document management.
• Edge devices and real-time applicationsSuitable for speed-sensitive scenarios such as mobile search, local retrieval on IoT devices, and high-concurrency real-time recommendations.
• Intelligent processing of long documentsThe context-aware version uses late chunking technology to generate semantically coherent paragraph embeddings for long texts such as legal contracts, academic papers, and technical documents, thereby improving the accuracy of long document retrieval.