Hy4-preview now features a GGUF quantized version, with a minimum of approximately 214 GiB.
AngelSlim has released a GGUF quantization repository for Hy4-preview, offering a Q4_K_M version with approximately 435 GiB and a mixed low-bit version with approximately 214 GiB, but requires a special llama.cpp patch to run.
Model publisher AngelSlim publicly released the GGUF quantized version of Hy4-preview on Hugging Face.The model card lists two main builds, including the Q4_K_M version at approximately 435.20 GiB and the STQ1_0 hybrid low-bit version at approximately 213.66 GiB, providing smaller options for local deployment of this 770B-class model.
The repository also states that these files cannot be run directly on the standard version of llama.cpp; they require applications provided by the repository.hyv4Architecture patch. This quantitative repository is not Tencent's official base model release page; therefore, the final version only attributes the quantitative work to the actual repository publisher.
refer to:Hugging Face Model Repository