Youtu-LLM - A lightweight language model open-sourced by Tencent Youtu team.

What is Youtu-LLM?

Youtu-LLM is a lightweight language model open-sourced by Tencent Youtu team, with 1.96 billion parameters. Designed specifically for agent tasks, it possesses powerful "native agent capabilities," outperforming models of similar or even larger scale in multiple tasks. The model employs a compact architecture and a 128K long context window, supporting long-range task processing, and features a vocabulary optimized for STEM domains to improve inference efficiency. The model is fully open-source, supporting both Base and Instruct versions, and provides fine-tuning tools for easy developer use.

Main functions of Youtu-LLM

• powerful intelligent agent capabilitiesIt possesses native intelligent agent capabilities, enabling it to autonomously plan tasks, execute operations, and adjust strategies based on feedback, making it suitable for handling complex tasks.
• Efficient reasoning and long context supportIt features a compact architecture and supports 128K context windows, making it suitable for long-running tasks such as complex code fixing and multi-hop research.
• Optimized STEM vocabularyDesigned for professional fields such as mathematics and coding, it improves token compression rate and inference efficiency.
• Systematic pre-training courses: Start from scratch and pre-train, and gradually improve the model's capabilities through three stages of learning: common sense, STEM, and intelligent agent.
• High-quality intelligent agent trajectory dataIntroducing a large amount of agent trajectory data, covering scenarios such as mathematical reasoning, code repair, and in-depth research, to improve the model's performance in agent tasks.
• Open source and flexible deploymentIt is fully open source, supports both Base and Instruct versions, and provides fine-tuning tools to facilitate deployment by developers in resource-constrained scenarios.

The technical principles of Youtu-LLM

• Compact architecture and long contextIt adopts a Dense MLA architecture, supports 128K context windows, is suitable for long-term task processing, and maintains efficient inference capabilities.
• Optimized word segmenter designA 128K vocabulary designed specifically for STEM fields, improving reasoning efficiency in specialized areas such as mathematics and coding.
• Three-stage pre-training courseThe pre-training process starts from scratch and is divided into a common sense stage, a STEM-focused stage, and an agent-based training stage, gradually improving the model's reasoning and planning capabilities.
• Intelligent agent trajectory synthesisIntroducing high-quality agent trajectory data, covering scenarios such as mathematical reasoning, code repair, and in-depth research, to enhance the model's performance in agent tasks.
• Innovative training paradigmThrough curriculum-based training that progresses from "common sense → STEM → intelligent agent," the model internalizes planning, execution, and reflection capabilities, achieving lightweight yet native intelligence.

The project address for Youtu-LLM

• Project official websitehttps://youtu-tip.com/#llm
• Github repository: https://github.com/TencentCloudADP/youtu-tip/tree/master/youtu-llm

Application scenarios of Youtu-LLM

• Code AssistantAble to understand complex codebase structures, locate and fix errors in code, write code that conforms to project standards, and automate software engineering tasks.
• Research AssistantSuitable for literature reviews and information integration, it supports multi-hop question answering and knowledge reasoning, can automatically generate research reports, and assists in academic writing.
• General intelligent agentIt can automate daily tasks, enable multi-tool collaboration, break down and execute complex tasks, and be used as a personal AI assistant.
• Edge deploymentDue to its small parameter count, it is suitable for running on consumer-grade GPUs, mobile devices, and edge devices, providing low-latency, low-cost inference services, and is suitable for local deployment in privacy-sensitive scenarios.
• Multi-hop reasoning and summarizationIt can solve complex problems that require multi-step reasoning, such as deep correlation analysis and causal reasoning.
• Knowledge-intensive tasksIt can efficiently handle problems that rely on a large amount of structured knowledge, such as enterprise knowledge base Q&A and in-depth analysis of technical documents.