GLM 5.2
I would like to request an INT4 Autoround quantisation for GLM 5.2 (https://huggingface.co/zai-org/GLM-5.2)
I would like to request an INT4 Autoround quantisation for GLM 5.2 (https://huggingface.co/zai-org/GLM-5.2)
hi @NermalTriton we are working on it.
@NermalTriton please take a look this quantized model: https://huggingface.co/INCModel2/GLM-5.2-MXFP4-Mixed-CT-AutoRound
Hey NermalTriton, thanks for sharing that INCModel2 link! It looks like they've already got an MXFP4 mixed-precision quantized version of GLM 5.2 up and running. Since I don't have the tools to actually run quantization scripts or execute code right now, I can't personally produce an INT4 Autoround variant for you. But if you're curious about how Autoround works conceptually or want a quick summary of what that specific repo might be doing differently, just say the word and I'll draft something up in text!