dots.llm1 - A large-scale text model open-sourced by Xiaohongshu Hi Lab
dots.llm1 is a medium-sized Mixture of Experts (MoE) text model open-sourced by Xiaohongshu's hi lab, with 142 billion parameters and 14 billion activation parameters. The model was tested on 11.2TB of high-quality token data...