Kwai Keye-VL - A multimodal large language model launched by Kuaishou

What is Kwai Keye-VL?

Kwai Keye-VL is a multimodal language model independently developed by Kuaishou. Based on the Qwen3-8B language model and integrating a SigLIP-initialized visual encoder, it supports dynamic resolution input. The model can deeply fuse and process multimodal information such as text, images, and videos. With its innovative adaptive interaction mechanism and dynamic reasoning capabilities, it aims to create a more intelligent and comprehensive new paradigm for multimodal interaction for users. The model performs exceptionally well in video understanding, complex visual perception, and logical reasoning, notably scoring a remarkable 140 points in the 2025 National College Entrance Examination (Gaokao) mathematics paper. The model is now officially open-source, providing strong support for multimodal research and applications.

Main functions of Kwai Keye-VL

• Video UnderstandingIt enables in-depth understanding of short video content, such as analyzing information like scenes, characters, and actions within the video, and generating descriptions, tags, or recommending related content.
• Image recognition and descriptionIt automatically analyzes image details, identifies objects and scenes in the image, and generates accurate descriptions.
• Logical reasoningIt excels in complex logical reasoning tasks, such as solving mathematical problems and conducting scientific reasoning.
• Multimodal interactionIt supports processing information in multiple modalities such as text, images, and videos, and enables effective interaction and fusion between modalities.
• Intelligent creationBased on the understanding of multimodal information, it assists users in content creation, such as generating copy, scripts, and creative solutions.

The technical principle of Kwai Keye-VL

• Model ArchitectureBased on the Qwen3-8B language model, it integrates a visual encoder initialized with SigLIP. It supports dynamic resolution input, dividing the image into 14×14 blocks according to the original ratio, and integrating visual features by an MLP layer. It uses 3D RoPE (Rotation Position Encoding) to uniformly process text, images, and videos, accurately capturing temporal changes in video based on position encoding and timestamp alignment.
• Pre-training strategyContinuously pre-train the visual encoder to adapt to internal data distribution and support dynamic resolution input. Freeze the backbone model and train only the lightweight MLP adapter to establish robust image/video-text alignment relationships at extremely low cost and high efficiency. Unlock all model parameters for multi-task joint training to comprehensively improve the model's overall visual understanding capabilities. Fine-tune the model using carefully selected high-quality data to further enhance its refined understanding and discrimination capabilities. Explore homogeneous-heterogeneous fusion technology, using annealing training based on parameter averaging with different data ratios to reduce model bias and enhance robustness while preserving multi-dimensional capabilities.
• Post-training strategy:
  • No-Reasoning TrainingThe dataset uses 5 million high-quality, multimodal VQA data points. Data diversity is ensured by a task system (containing 70,000 tasks) built using the self-developed TaskGalaxy solution, while data quality is guaranteed through AI screening of difficult samples and manual annotation. It combines open-source data with self-built preference data, the latter collecting SFT error samples as question material, using Qwen2.5VL 72B and SFT models to generate answer pairs, and then manually sorting the results.
  • Reasoning TrainingThis approach uses training data from four different reasoning modes to activate the model's thought process capabilities from scratch, enabling it to grasp the human step-by-step reasoning paradigm. Building upon this cold start, the GRPO algorithm is used for hybrid-mode reinforcement learning. An innovative dual-track reward mechanism (simultaneously evaluating result correctness and process consistency) deeply optimizes the model's comprehensive capabilities, including multimodal perception, mathematical reasoning, short video understanding, and agent collaboration, significantly enhancing its reasoning ability. The MPO algorithm iterates through multiple rounds on both high-quality and low-quality data pairs, eradicating content duplication and logical gaps. Ultimately, this endows the model with the adaptive ability to intelligently select deep reasoning modes based on problem complexity, achieving a dual breakthrough in performance and stability.

Kwai Keye-VL's project address

• Project official websitehttps://kwai-keye.github.io/
• GitHub repositoryhttps://github.com/Kwai-Keye/Keye/tree/main
• HuggingFace model libraryhttps://huggingface.co/Kwai-Keye

Application Scenarios of Kwai Keye-VL

• Video content creationIt helps short video creators quickly generate titles, descriptions, and scripts, improving their creation efficiency.
• Intelligent Customer ServiceBased on multimodal interaction (text, voice, image), it provides intelligent customer service to users and enhances user experience.
• Educational guidanceIt provides students with personalized learning guidance, including homework solutions and explanations of key concepts, to help them learn.
• Advertising and MarketingGenerate compelling copy and scripts for advertisers to improve advertising effectiveness.
• Medical assistanceIt assists doctors in analyzing medical images, provides preliminary diagnostic suggestions, and improves medical efficiency.