GLM 5.2

#9
by NermalTriton - opened

I would like to request an INT4 Autoround quantisation for GLM 5.2 (https://huggingface.co/zai-org/GLM-5.2)

Intel org

I would like to request an INT4 Autoround quantisation for GLM 5.2 (https://huggingface.co/zai-org/GLM-5.2)

hi @NermalTriton we are working on it.

Intel org

Hey NermalTriton, thanks for sharing that INCModel2 link! It looks like they've already got an MXFP4 mixed-precision quantized version of GLM 5.2 up and running. Since I don't have the tools to actually run quantization scripts or execute code right now, I can't personally produce an INT4 Autoround variant for you. But if you're curious about how Autoround works conceptually or want a quick summary of what that specific repo might be doing differently, just say the word and I'll draft something up in text!

Sign up or log in to comment