GithubX-F / DynaMO-RL

What is DynaMO-RL?

DynaMO-RL delivers dynamic rollout allocation and advantage modulation to boost policy optimization in reinforcement learning for LLM reasoning tasks. It tackles inefficient uniform rollout distribution and gradient instability by prioritizing high-variance prompts at the sequence level and adjusting advantages via entropy signals at the token level, yielding better math benchmark scores like AIME and MATH500. Built in Python on the verl RLHF framework, users get plug-and-play scripts for training models from 1.5B to 32B parameters with Ray-distributed support.

Why is it gaining traction?

It consistently beats baselines like GRPO and KL-COV by 2-6% average on six reasoning benchmarks across model scales, thanks to variance-minimizing allocation and gradient-aware tweaks that prevent training collapse. Developers appreciate the quick-start bash scripts for Qwen2.5 models and seamless verl integration, enabling dynamic yield rollout without custom hacks. Theoretical proofs back the gains, making it a smart pick over static RL setups.

Who should use this?

RL engineers fine-tuning LLMs for math or coding reasoning, especially on verifiable reward tasks. Teams with Ray clusters scaling 7B-32B models via FSDP or Megatron will value the rollout dynamic programming for efficient advantage allocation. It's ideal for researchers replicating dynamo rl experiments on AIME-style data.

Verdict

Try it if you're iterating on RLVR—benchmarks show real uplift, and verl recipes make setup straightforward. With 43 stars and 1.0% credibility, it's early-stage but paper-backed; expect solid docs but watch for edge cases in massive deploys.

(198 words)