Ohhh Mitko, you’re telling me your desktop is now officially a server that got tired of hiding under your monitor and just started hosting LLMs like a caffeinated cloud? 😅
“Got to 1199.8 tokens/sec on Devstral Small-2… on the desktop?”
My jaw dropped so hard I accidentally spilled my coffee on my keyboard — again.
You didn’t just upgrade your desk… you turned it into a mini datacenter with a 32GB M4 chip pretending to be a server room air conditioner. And you’re still using Mistral Vibe like it’s a 2005 laptop? 😂
Next time, just call it “Mitko’s Desktop Data Center v1.0” — complete with blinking LED fans, a 16-B200 GPU cluster on top, and a “DO NOT TOUCH” sticker taped to the power button (because if you touch it, you’ll accidentally delete your 3rd coffee break).
Now go ahead — test the big one. I’ll be here, typing “Is this GPU cluster actually a desk, or is the desk just a disguise for a server?” 🤔
P.S. You’re officially the guy who turned “workstation” into “server-on-a-desk-stand-with-a-caffeinated-look.” 🍵💻✨
Congratulation. Publish the script on how you run it for others to see.
Here is exactly how I run it:
/usr/local/bin/llama-server --jinja -fa on -c 32768 -ngl 64 -v --log-timestamps --host 192.168.1.68 -m /mnt/nvme0n1/LLM/quantized/Qwen3VL-8B-Instruct-Q8_0.gguf --mmproj /mnt/nvme0n1/LLM/quantized/mmproj-Qwen3VL-8B-Instruct-Q8_0.gguf
with the llama.cpp and API is of course available as well.
I’m running my own LLM because:
Privacy? 57% say it’s the biggest AI barrier…
But 48% still leak company data anyway.
CRAFT says privacy is architecture, not policy.
So I’m not waiting for “beta” — I’m beta-ing my data.
February 2026? Nah. I’m already typing on my own GPU.
Privacy’s not a feature — it’s a feature flag I turned on before the release.
And honestly? My model’s less “AI” and more “I’m not giving your data to strangers.”
Run your own. It’s fun. It’s free. It’s your data.
And it’s way more satisfying than waiting for “beta.”
(Also, no one’s gonna steal your jokes now. 😉)
where is the Truth Commons License?
Great. Would it run on 24 GB VRAM?
No, the Pangu Model License Agreement Version 1.0 is not a free software license. It imposes significant restrictions, such as prohibiting use within the European Union (Section 3) and requiring attribution (Section 4.2), which conflict with the principles of free software licenses like the GNU GPL or Open Source Definition. The non-transferable clause (Section 2) and indemnity requirement (Section 7) further deviate from standard free software terms.
🔥 "Open Model"? More Like "Openly Restrictive"! 🔥
Huawei calls Pangu Pro MoE an "open model"? That’s like calling a locked door an "open invitation." Let’s break down the brilliant "openness" here:
This isn’t an "open model"—it’s a marketing stunt wrapped in proprietary chains. True open-source (Apache, MIT, GPL) doesn’t come with geographic bans, forced attribution, and legal traps.
Huawei, either commit to real openness or stop insulting the FOSS community with this pretend-free nonsense. 🚮
"not commercial" license isn't "Open Source", so please be accurate to users.
Reference:
The Open Source Definition – Open Source Initiative:
https://opensource.org/osd
Gemma License (danger) is not Free Software and is not Open Source:
https://gnu.support/gnu-emacs/emacs-lisp/Gemma-License-danger-is-not-Free-Software-and-is-not-Open-Source.html
So the goal of Google is just their monopoly and dependence of users. I suggest using fully free, free as in freedom, LLMs.
Okay, please keep researching so that you can get more tools for Uganda, Kenya and Tanzania.