ByungKwanLee / Distill-R1

What is Distill-R1?

Distill-R1 is a Python-based open-source RL framework for training vision-language models with online teacher-student distillation. It lets you run reinforcement learning while a fixed teacher model generates rollouts and transfers knowledge via KL or JSD losses on both student and teacher data, boosting reasoning in smaller models like deepseek r1 distill qwen 14b or qwen 32b from teachers like deepseek r1 distill llama 70b. Users get quick multi-node setups via Ray, FSDP sharding, and vLLM engines, plus bash scripts for single-node starts and model merging.

Why is it gaining traction?

Unlike standard RL tools that train isolated policies, Distill-R1 adds distillation without heavy rewrites—it's a minimal fork of EasyR1, making diffs easy to audit. Developers dig the supported algos like GRPO and DAPO on Qwen VL models, plus metrics logging for teacher KL alignment and throughput. As an llm open source framework, it hooks ai agents framework open source builders needing efficient distillation baselines.

Who should use this?

AI researchers distilling reasoning chains into VLMs for tasks like visual QA or multimodal agents. Teams fine-tuning qwen variants or deepseek-r1-distill-gwen setups on custom datasets, especially with multi-GPU clusters. Not for prod deploys, but ideal for github open source tools explorers prototyping ollama r1-distill workflows.

Verdict

Promising research baseline for distill r1 deepseek experiments, with solid docs and quickstarts, but 19 stars and 1.0% credibility signal early maturity—expect bugs in edge cases. Grab it if you're hacking open source framework list entries like this for VL RLHF.

(198 words)