Tencent releases Hy4 preview and opens up model weights
Tencent Hy team released Hy4 preview, which adopts the MoE architecture with a total of 770 billion parameters and 49 billion parameters activated per token, supports 1 million token contexts, and opens up standards and FP8 weights.
Tencent Hy team released Hy4 preview and made the model weights available for both standard and FP8 versions.The model employs a hybrid expert architecture, containing a total of 770 billion parameters, with each token activating 49 billion parameters, and supports 1 million token contexts.
The project is licensed under the Apache 2.0 license. The team also emphasized that this is still an early preview version, and known limitations have been disclosed. Official evaluation results should be interpreted according to the vendor's own testing.
refer to:Tencent Hy Research Page、Official GitHub repository