I alredy do some from scratch, i basicly have a unlimited 4090 with solar what i can use durning day lol

The H200 vs B200 line is the part I would push on. A 15B token from-scratch run that small usually stops being compute bound well before it stops being dataloader bound. So a cost benchmark across two GPU tiers can quietly end up measuring your input pipeline instead of the silicon.

Did you log MFU or GPU utilization on both? If B200 utilization came in lower than H200, the $/token gap is a plumbing number, not a hardware one, and that changes the advice for anyone replicating.

Also curious what the $11 run actually validated for you. Loss curve shape, or an eval you already trusted?

Great article!

One thing worth noting -> it opens and closes the reasoning tag due to the missing enable_thinking flag in your tokenizer's apply_chat_template during generation.

text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

It might or might not learn anything special inside the reasoning part, but you forced it not to show it :P

Qwen3.5 0.8B chat_template:

{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
    {%- if enable_thinking is defined and enable_thinking is true %}
        {{- '<think>\n' }}
    {%- else %}
        {{- '<think>\n\n</think>\n\n' }}
    {%- endif %}
{%- endif %}