verl-project / verl-omni

What is verl-omni?

VeRL-Omni is a Python RL training framework for diffusion models generating images, videos, or audio, and omni-modality models handling text, image, video, and audio together—like Qwen3-Omni or verl qwen omni setups. Built on PyTorch and verl as a common training framework, it streamlines RL post-training for multimodal generative models, tackling I/O bottlenecks and compute hurdles that plague text-only LLM workflows. Users get high-throughput rollouts via vLLM-Omni integration, flexible reward pipelines, and end-to-end configs for algorithms like FlowGRPO.

Why is it gaining traction?

It delivers 25% higher end-to-end throughput than diffusers-based alternatives on Qwen-Image FlowGRPO benchmarks, thanks to async rewards, FSDP backends, and specialized multimodal sampling. Developers hook into modular pipelines for custom rewards—rule-based or model-driven—without rebuilding parallelism stacks. As a GitHub repo training tool with quickstarts and docs, it cuts setup time for pytorch training framework experiments on private repos or github data training.

Who should use this?

AI engineers fine-tuning diffusion models with RLHF-style methods, like image/video generators needing reward-driven alignment. Teams training framework llm extensions for multimodal tasks, or exploring training framework examples on Qwen-Image/Wan2.2. Suited for those scaling github training on private repos beyond basic diffusers scripts.

Verdict

Try it for early multimodal RL prototypes—solid docs and benchmarks make it accessible despite 19 stars and 1.0% credibility score signaling immaturity. Pair with verl for production; lacks broad tests but shines in niche high-throughput needs.

(198 words)