HY-1.8B-2Bit - Tencent Hunyuan's first industry-grade 2Bit edge-side model

What is HY-1.8B-2Bit?

HY-1.8B-2Bit is Tencent Hunyuan's first industry-grade 2-bit edge AI model, based on the 1.8B model and compressed using Quantization-Aware Training (QAT). HY-1.8B-2Bit has an equivalent parameter count of only 0.3B, a size of only 300MB, and a memory footprint of 600MB, smaller than most mobile applications. While maintaining the original model's full thinking capabilities, it improves generation speed by 2-3 times. It is already adapted to mobile platforms such as Arm SME2 and can run efficiently on edge devices such as mobile phones, headphones, and smart home devices, achieving a breakthrough in "small yet powerful" edge AI deployment.

Main functions of HY-1.8B-2Bit

• Edge-side intelligent inferenceThe model supports offline deployment and operation of large language models on consumer-grade hardware such as mobile phones, headphones, and smart home devices, without relying on cloud services.
• Full thinking abilityIt retains the original model's flexible switching mechanism between long and short thought chains, automatically selecting a concise or detailed reasoning path based on task complexity.
• High-efficiency text generationCompared to the original precision model, it achieves a 2-3 times speed improvement in generation on real edge devices, significantly reducing user waiting time.
• Privacy and security protectionThe model runs entirely locally, ensuring that user data does not leave the device, thus meeting the security requirements of offline deployment and highly private scenarios.

Technical principles of HY-1.8B-2Bit

• Quantization-Aware Training (QAT)Unlike traditional post-training quantization (PTQ), which results in significant accuracy loss due to direct compression after model training, QAT simulates the forward propagation effect of 2-bit quantization during training, enabling model weights to proactively adapt to low-precision constraints and significantly restore the performance of the quantized model.
• 2-bit extreme compression and equivalent parameter reductionBy compressing 32-bit floating-point weights to 2-bit representation (only 4 discrete states), the actual model size is reduced from GB level with 1.8B parameters to 300MB, with an equivalent parameter quantity of only 0.3B, achieving a 6x storage compression ratio, while maintaining mathematical, code, and scientific reasoning capabilities comparable to 4-bit PTQ.
• Data and training strategy optimizationThe model employs three methods—data optimization, elastic stretching quantization, and innovative training strategies—to collaboratively enhance its overall capabilities. Among these, elastic stretching quantization dynamically adjusts the quantization range of the weight distribution to minimize the accuracy loss caused by extreme values.
• End-side hardware adaptationIt provides GGUF-INT2 format weights and BF16 pseudo-quantized weights, and performs deep optimization for the Arm SME2 instruction set, enabling the model to achieve efficient and stable inference on mobile devices that enable this technology.

Project address for HY-1.8B-2Bit

• GitHub repositoryhttps://github.com/Tencent/AngelSlim
• HuggingFace model library:
  • https://huggingface.co/AngelSlim/HY-1.8B-2Bit-GGUF
  • https://huggingface.co/AngelSlim/HY-1.8B-2Bit
• Technical Papers: https://huggingface.co/AngelSlim/HY-1.8B-2Bit/blob/main/AngelSlim_Technical_Report.pdf

Application scenarios of HY-1.8B-2Bit

• Smartphone AssistantAs a built-in AI assistant, it enables offline voice interaction, text generation, and intelligent question answering. It can handle daily tasks such as schedule management and email drafting without an internet connection, protecting user privacy while reducing data consumption.
• Smart wearable devicesThe model can be deployed on low-power devices such as smart headphones and watches, supporting offline speech-to-text, real-time translation, and health consultation, solving the pain points of limited computing power and battery life sensitivity of wearable devices.
• Smart Home Central ControlIt runs locally on IoT devices such as smart speakers and home appliance controllers, enabling offline voice command recognition, scene linkage control, and personalized recommendations, improving response speed and ensuring home data security.
• Vehicle edge computingIt is used in vehicle infotainment systems and autonomous driving assistance devices to ensure service continuity in scenarios with unstable networks.
• Enterprise privatization deploymentThe model can meet the needs of industries with extremely high data compliance requirements, such as finance, healthcare, and government, and can handle sensitive documents and answer professional questions on local servers or dedicated terminals, eliminating the risk of data leakage.