0xD4rky / Tiny-RL

What is Tiny-RL?

Tiny-RL is a Python toolkit for training small LLMs like Qwen 1.7B on math problems using RL methods such as GRPO, DAPO, and REINFORCE++. It generates completions via vLLM for fast rollouts, scores them automatically against ground-truth answers, and fine-tunes the model with configurable hyperparameters in a YAML file. Run it with `uv sync` then `uv run train.py --config config.yaml`, saving checkpoints to `./output` and logging metrics to wandb—perfect for quick RL experiments on reasoning tasks.

Why is it gaining traction?

It stands out by packing multiple state-of-the-art RL losses into a minimal setup, letting you swap algorithms like grpo or dapo without rewriting code, unlike heavier frameworks. Developers dig the vLLM integration for speedy inference on limited GPUs and the built-in math reward model that handles boxed answers and normalization. As a compact scripts repo akin to proxmox scripts repo or react scripts repo, it hooks RL tiny gaston fans wanting reproducible math benchmarks without the bloat.

Who should use this?

RL engineers benchmarking LLM alignment on datasets like competition_math, or AI researchers prototyping math reasoning boosts on consumer hardware. Ideal for indie devs forking repo github private setups to test custom losses, or teams using repo github actions for CI/CD on tiny house-scale models—skip if you're doing non-math domains.

Verdict

Grab it for fast RLHF prototyping on math tasks if you're okay with its 1.0% credibility score and 19 stars signaling early-stage maturity—docs are basic but setup is dead simple. Not battle-tested for production, but a solid starting point for experiments.

(178 words)