zero-gpu-explorers (ZeroGPU Explorers)

posted an update about 6 hours ago

Post

42

TRL now includes agent training support for GRPO‼️

Train 🕵️ agents with 🔧 tools, enabling interaction with external functions and APIs.

And of course, a new notebook and scripts to get you up to speed

📘 notebook tutorial: https://github.com/huggingface/trl/blob/main/examples/notebooks/grpo_agent.ipynb

📂 script examples: https://github.com/huggingface/trl/blob/main/examples/scripts/grpo_agent.py

📦 TRL v0.26.0 release: https://github.com/huggingface/trl/releases/tag/v0.26.0

sergiopaniego

posted an update 1 day ago

Post

1402

ICYMI, you can fine-tune open LLMs using Claude Code

just tell it:
“Fine-tune Qwen3-0.6B on open-r1/codeforces-cots”

and Claude submits a real training job on HF GPUs using TRL.

it handles everything:
> dataset validation
> GPU selection
> training + Trackio monitoring
> job submission + cost estimation
when it’s done, your model is on the Hub, ready to use

read more about the process: https://huggingface.co/blog/hf-skills-training

IliaLarchenko

posted an update 1 day ago

Post

124

🏆 BEHAVIOR Challenge 1st Place – Solution Summary

My team recently won 1st place in the BEHAVIOR Challenge at NeurIPS.
The competition focused on training a single policy to complete 50 long-horizon household tasks in simulation.

We built an end-to-end policy based on Pi0.5 with a bunch of custom modifications. Everything is open-sourced, and it should be useful for anyone exploring VLAs or adapting them to specific tasks.

Key Architecture Changes:
- Replaced language model with 50 trainable task embeddings (no text at all)
- Correlated noise for Flow Matching: ϵ ∼ N(0, 0.5I + 0.5Σ) using dataset action covariance
- Learnable mixed-layer attention: each action expert layer attends to a trainable mix of all VLM layers
- System 2 stage tracking: model predicts task stage, we smooth it with voting and feed it back as context

Training:
- Multi-sample Flow Matching: 15 FM samples per VLM pass to reduce gradient variance
- Delta action space + per-timestamp normalization
- FAST auxiliary loss and stage prediction loss
- Trained on 224×224 RGB + proprioception only
- We use 4 fine-tuned checkpoints, all derived from a multi-task model trained on all 50 tasks

Inference Optimizations:
- Soft inpainting: predict 30 actions, execute 26, use 4 as an input for the next chunk
- Correlation-aware guidance of inpainting to keep action chunks smooth
- 1.3× speedup via cubic spline compression
- General correction rule: reopen gripper after failed grasps

🔗 Code and Models:
- Code: https://github.com/IliaLarchenko/behavior-1k-solution
- Weights: IliaLarchenko/behavior_submission
- Paper: Task adaptation of Vision-Language-Action model: 1st Place Solution for the 2025 BEHAVIOR Challenge (2512.06951)

sergiopaniego

posted an update 1 day ago

Post

1355

We just released TRL v0.26.0!

It comes packed with updates:
> Agent training with tools in GRPO
> New CISPO & SAPO losses + reasoning rewards
> vLLM quantization in colocate mode
> Dataset shuffling in SFT
> Lots of NEW examples
> Tons of fixes and documentation improvements

3 replies

·

IliaLarchenko

authored a paper 2 days ago

Task adaptation of Vision-Language-Action model: 1st Place Solution for the 2025 BEHAVIOR Challenge

Paper • 2512.06951 • Published 4 days ago

sergiopaniego

posted an update 2 days ago

Post

2742

NEW: @EssentialAI just released Rnj-1, their first 8B model.

You can easily fine-tune it with GRPO using TRL to add reasoning capabilities to a compact mode

Free Colab link: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_rnj_1_instruct.ipynb

More free TRL notebooks: https://huggingface.co/docs/trl/main/en/example_overview#notebooks

sergiopaniego

posted an update 6 days ago

Post

2778

Want to get started with fine-tuning but don’t know where to begin? 🤓☝️

We’re expanding our collection of beginner-friendly free Colab notebooks so you can learn and fine-tune models using TRL at no cost

🔬 Check out the full list of free notebooks: https://huggingface.co/docs/trl/main/en/example_overview#notebooks

🔬 If you want more advanced content, we also have a lot to cover in the community tutorials: https://huggingface.co/docs/trl/community_tutorials

And now the obvious question: what would you like us to add next?

sergiopaniego

posted an update 8 days ago

Post

2320

NEW: @mistralai released a fantastic family of multimodal models, Ministral 3.

You can fine-tune them for free on Colab using TRL ⚡️, supporting both SFT and GRPO

Link to the notebooks:
- SFT: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/sft_ministral3_vl.ipynb
- GRPO: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_ministral3_vl.ipynb
- TRL and more examples: https://huggingface.co/docs/trl/index

2 replies

·

sergiopaniego

posted an update 10 days ago

Post

2146

ICYMI, transformers v5 is out!

Grab a coffee ☕ and go read the announcement blog https://huggingface.co/blog/transformers-v5

zubingou

authored a paper 10 days ago

DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning

Paper • 2511.22570 • Published 14 days ago • 71

sergiopaniego

posted an update 10 days ago

Post

3080

want to use open models easily through an API?

Inference Providers might be exactly what you’re looking for sooo here’s a complete beginner-friendly walkthrough 🧐

https://www.youtube.com/watch?v=oxwsizy1Spw

2 replies

·

BK-Lee

authored 2 papers 10 days ago

MultiVerse: A Multi-Turn Conversation Benchmark for Evaluating Large Vision and Language Models

Paper • 2510.16641 • Published Oct 18 • 4

RefineBench: Evaluating Refinement Capability of Language Models via Checklists

Paper • 2511.22173 • Published 15 days ago • 12

sergiopaniego

posted an update 14 days ago

Post

1728

nanochat is now in transformers!

The LLM by @karpathy is officially in the library, and we wrote a blog covering: how did we port the model, differences from the original, and how to run or train it.

go read it 🤓

nanochat-students/transformers

sergiopaniego

posted an update 16 days ago

Post

3948

you gotta go fast and go read the latest blog by @ror et al. explaining Continuous Batching in depth

https://huggingface.co/blog/continuous_batching

sergiopaniego

posted an update 17 days ago

Post

1712

Interested in RL training environments?

We just released a beginner-friendly walkthrough notebook!

Train a model to play Wordle using TRL + OpenEnv (TextArena) + GRPO + vLLM.

happy learning! 🌱

Notebook: https://github.com/huggingface/trl/blob/main/examples/notebooks/openenv_wordle_grpo.ipynb

OpenEnv guide in TRL: https://huggingface.co/docs/trl/main/en/openenv

sergiopaniego

posted an update 21 days ago

Post

328

Ya está disponible el vídeo de la charla del otro día en @nerdearla sobre IA abierta, por si queréis verla! 🤠

https://www.youtube.com/watch?v=p-JLn4xAkMw

1 reply

·

sergiopaniego

posted an update 22 days ago

Post

2581

we've just added several example scripts to TRL showing how to train models with GRPO using some of the new OpenEnv environments

train a model to interact with a browser (🎮 BrowserGym Env), play Wordle (🎮 Wordle Env) and moooore!

TRL (GRPO + vLLM) + OpenEnv! ⚡️

📝 go play with them: https://github.com/huggingface/trl/tree/main/examples/scripts/openenv

📝 examples list: https://huggingface.co/docs/trl/main/en/example_overview#scripts