LongCat-2.0-Preview - A preview version of Meituan's trillion-parameter-level large-scale model.

What is LongCat-2.0-Preview?

LongCat-2.0-Preview is a preview version of a trillion-parameter-scale large-scale model launched by Meituan. It adopts the MoE architecture, with a total of 1.6T parameters and approximately 48B activation parameters, and supports a 1M ultra-long context. The model completes the entire training and inference process based on 50,000-60,000 domestic accelerator cards, with NVIDIA's computing power accounting for 0%. It is the first time in the industry that a trillion-scale large-scale model training and inference closed loop has been successfully run on domestic chips. The model is currently open for internal testing, providing 10 million free tokens per day.

Main functions of LongCat-2.0-Preview

• Extremely long context understandingSupports 1M token long text input, and can handle massive amounts of content such as entire books, large code repositories, and long reports.
• Code generation and reasoningBased on the MoE architecture, the expert network is precisely scheduled and performs stably in programming, mathematical reasoning, and complex instruction following tasks.
• Multi-turn dialogue and semantic retrievalBy using sparse attention mechanisms, key information can be quickly located in long dialogue histories, reducing redundant calculations.
• Multimodal content understandingLeveraging Meituan's accumulated experience in local life scenarios, it supports in-depth analysis of complex business documents and mixed text and image content.

Technical Principles of LongCat-2.0-Preview

• MoE (Hybrid Expert) ArchitectureOf the total 1.6T parameters, only 48B are activated to participate in a single forward computation. The most relevant expert modules are dynamically selected through gating routing to reduce inference costs.
• N-gram EmbeddingBy moving some parameters that were originally located in the expert layer to the Embedding layer, and introducing phrase-level modeling capabilities, high-frequency language patterns can be directly matched and hit, reducing the computational overhead of each layer.
• Sparse attention and cross-laminar flow perception indexWe introduce a lightweight sparse attention mechanism and overlay cross-layer semantic path indexes to avoid full O(n²) attention computation and ensure that inference latency is controllable under 1M context.
• Deterministic operators developed in-houseTargeting the domestic chip ecosystem, we have independently developed FlashAttention Backward Gradient (FAG), Scatter-type operators, and the GEMM module, controlling the performance loss to around 5% and ensuring the reproducibility of long-term training on the Wanka cluster.
• Memory and communication optimizationThe peak training memory is compressed to below 60GB using the V-ZB algorithm, and the expert parallel (EP), tensor parallel (TP) and pipeline parallel (PP) strategies are reconstructed to adapt to the bandwidth and memory constraints of domestic chips.

How to use LongCat-2.0-Preview

• Apply for beta testing qualificationVisit the LongCat Open Platform, click on the internal testing application entry on the official website, fill in the basic information and submit for review.
• Awaiting approvalThe team will review the eligibility of the application after submission, and notify the user of the activation of permissions via email or SMS once approved.
• Log in to the platform to obtain credit limitLog in to the LongCat platform with an approved account, and the system will automatically issue a daily free quota of 10 million tokens.
• Select access methodUsers can experience it directly through the web chat interface, or obtain the API Key according to the documentation to integrate the model into their own applications and workflows.
• Start Long Text TaskEnter long documents, codebases, or complex instructions in a dialog box or API request, and the model will perform inference with the MoE expert network based on the 1M context window and return the results.