Instructions to use tchbcb/samai-27b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tchbcb/samai-27b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tchbcb/samai-27b", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("tchbcb/samai-27b", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tchbcb/samai-27b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tchbcb/samai-27b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tchbcb/samai-27b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tchbcb/samai-27b

SGLang

How to use tchbcb/samai-27b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tchbcb/samai-27b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tchbcb/samai-27b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tchbcb/samai-27b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tchbcb/samai-27b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tchbcb/samai-27b with Docker Model Runner:
```
docker model run hf.co/tchbcb/samai-27b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

samai-27b · PonderNet + dMoE + MTP (Qwen3.8-27B backbone)

将 samai-4b (tchbcb/samai-4b) 的自适应计算改造方案移植到 Qwen/Qwen3.8-27B 的 27B 因果/图文语言模型。骨干权重原样保留, 新增模块按稠密等价原则初始化 (零 router = 初始与基座逐位可对拍), 可直接进入下游 SFT。

架构

骨干: Qwen3.8-27B — 64 层混合 (48 gated-delta-net 线性注意力 + 16 GQA 全注意力, 每 4 层 1 个), hidden 5120, MLP 17408, vocab 248320, MPE 262144, 原生多模态包装 (SigLIP 视觉塔 27 层, 原样保留 → samai-27b 仍是图文模型)
PonderNet: 最后 8 个全注意力层 (L35/39/43/47/51/55/59/63) 各自为 ponder 位点: 层内 learned-halting 循环 (独立停机头, bias +4.0 → 初始 λ≈0.982 ≈ 2 步), 思考迭代用"替换式 KV"语义 (裁剪自家 KV 槽位重写), 终止后 commit pass 用混合表示重写 KV。刻意只选全注意力层 — 线性注意力层的 delta-rule 递归状态不可重入, 层级 ponder 完全绕开该状态, 实现零状态风险
dMoE: 8 个 ponder 层的 MLP 各拆 8 专家 (宽度均分 2176) + 零初始化 router (p=1/8 全激活 = 稠密等价起步), Switch 式负载均衡 (aux = N·Σf_i·P_i) + STE 梯度, tau=0.125
MTP: 继承基座原生 DeepSeek-V3 式 MTP 权重 (fc 2H→H + pre_fc_norm_hidden/ embedding + 1 层全注意力 decoder + norm), 训练期位置 t 输入 [h_t, e_{t+1}] 预测 t+2 (CE, mtp_loss_weight=1), 推理自动旁路

冒烟验证 (迷你随机权重, 26/26 PASS)

等价性: forced_steps=1 + 零 router 与原生 Qwen3.8 实现逐位对拍 — argmax 全对, maxdiff 1.0e-06 (fp32 GEMM 噪声), greedy 生成 token 序列完全一致
训练路径: 主 loss + ponder KL + MoE aux + MTP CE 全部有限; router/halter/ expert/MTP 梯度全非零
存取闭环: save_pretrained → trust_remote_code 重载, logits 逐位一致 (0.00e+00)

使用

from transformers import AutoModelForImageTextToText, AutoTokenizer
import torch
m = AutoModelForImageTextToText.from_pretrained(
    "tchbcb/samai-27b", trust_remote_code=True,
    dtype=torch.bfloat16, device_map="auto")
tok = AutoTokenizer.from_pretrained("tchbcb/samai-27b")
out = m.generate(**tok.apply_chat_template(
    [{"role": "user", "content": "你好"}], add_generation_prompt=True,
    tokenize=True, return_dict=True), max_new_tokens=256)
print(m.ponder_stats())   # 每层 executed 步数 / 平均思考步数 / MoE 激活专家数

AutoModelForCausalLM / AutoModel 亦注册到同一实现 (trust_remote_code=True)
运行期可调: m.config.ponder_forced_steps=1 (等价模式/直通), max_ponder_steps, commit_kv 等

训练 (SFT)

PonderNet: ponder_loss_beta KL 正则 (Geometric 先验), 停机头 bias +4 起步
dMoE: moe_aux_weight 负载均衡; router 从零起步, SFT 后稀疏化 (4b 实测 8→1.4~4.8)
MTP: mtp_loss_weight 控制 t+2 预测项 (推理旁路, 仅辅助训练)
注意: 27B LoRA SFT 需多卡或 QLoRA (T4 16GB 不可行); 训练时需开 gradient_checkpointing ( ponder 循环使激活 ≈ K 倍)

权重手术说明

19 个 safetensors 分片: 基座 18 (mlp→experts 键替换) + extras 1 (零 router ×8 + halter ×8, seed 42)
转换脚本 s27_convert.py (流式: 逐分片下载→手术→上传→删除, 断点续传)
冒烟脚本 s27_smoke.py; 建模代码 modeling_samai_27b.py (单文件, 依赖 transformers ≥5.8 的 qwen3_5 原生实现)

致谢与谱系

samai-2b (r18) → samai-4b (XHToken/Spark-X2.5-4B) → samai-27b (Qwen3.8-27B); PonderNet 思想源自 PonderNet/ACT, MoE 路由为 Switch 式, MTP 结构对齐 DeepSeek-V3。

Downloads last month: 25

Safetensors

Model size

28B params

Tensor type

BF16

Model tree for tchbcb/samai-27b

Base model

Qwen/Qwen3.8-27B

Finetuned

(776)

this model

Quantizations

1 model