Hugging Face's logo
Activity Feed

AI & ML interests

None defined yet.
yanghaojin 
posted an update 2 days ago
view post
Post
2,926
Dear all,
we are happy to share that gbx v0.8.0, our local inference engine for Apple Silicon, runs the 105 GB Qwen3.8-Flash-Next on a 48 GB MacBook Pro by streaming MoE experts from the SSD. When the model fits in memory, it decodes up to 50% faster than omlx, Ollama and mlx-lm on the same Mac, with output identical token for token.
Full write-up: https://medium.com/@semamediadata/running-a-105-gb-qwen-on-a-48-gb-macbook-pro-f116fe493a49
Download gbx: https://github.com/GreenBitAI/gbx-lm/releases
Models: https://huggingface.co/collections/GreenBitAI/greenbitai-for-apple-silicon
  • 2 replies
·
JOtholt 
updated a model 9 months ago
JOtholt 
published a model 10 months ago
yanghaojin 
posted an update about 1 year ago
yanghaojin 
posted an update about 1 year ago
view post
Post
194
Our new blog post Smaller Models, Smarter Agents 🚀 https://huggingface.co/blog/yanghaojin/greenbit-3-bit-stronger-reasoning
DeepSeek’s R1-0528 proved that 8B can reason like 235B. Anthropic showed that multi-agent systems boost performance by 90%. The challenge? Both approaches burn massive compute and tokens.
💡 GreenBitAI cracked the code:
We launched the first 3-bit deployable reasoning model — DeepSeek-R1-0528-Qwen3-8B (3.2-bit).
✅ Runs complex multi-agent research tasks (e.g. Pop Mart market analysis)
✅ Executes flawlessly on an Apple M3 laptop in under 5 minutes
✅ 1351 tokens/s prefill, 105 tokens/s decode
✅ Near-FP16 reasoning quality with just 30–40% token usage
This is how extreme compression meets collaborative intelligence — making advanced reasoning practical on edge devices.
yanghaojin 
posted an update over 2 years ago
view post
Post
925
Dear community,

Please check our recent blog post, "GPU Poor Savior: Revolutionizing Low-Bit Open Source LLMs and Cost-Effective Edge Computing". A cheaper and more efficient SFT scheme for quantized LLMs is provided.

https://huggingface.co/blog/NicoNico/green-bit-llm
yanghaojin 
posted an update over 2 years ago
view post
Post
2,319
Full parameter fine-tuning of the LLaMA-3 8B model using a single GTX 3090 GPU with 24GB of graphics memory?

Please check out our tool for fine-tuning, inferencing, and evaluating GreenBitAI's low-bit LLMs:
https://github.com/GreenBitAI/green-bit-llm
Model Zoo:
GreenBitAI
  • 3 replies
·
yanghaojin 
posted an update over 2 years ago
view post
Post
1,360
Dear all,

We are happy to share that we have just open-sourced over 200 low-bit LLMs. For the MLX community, we have prepared 2-4 bit versions of mainstream LLMs. You can visit the following collection to access them: https://huggingface.co/collections/GreenBitAI/greenbitai-mlx-llm-6614eb6ceb8da657c2b4ed58.

These low-bit models can be conveniently used through our open-source tool at https://github.com/GreenBitAI/gbx-lm.

Compared to other open-source quantization algorithms, these models provide better accuracy retention. We have provided some model evaluation results here:
https://github.com/GreenBitAI/green-bit-llm/blob/main/green_bit_llm/evaluation/README.md.

You can also evaluate the models yourself using the evaluation script we provided.
  • 1 reply
·
yanghaojin 
authored a paper about 3 years ago