GithubX-F / ProxMO-RL

What is ProxMO-RL?

ProxMO-RL is the official Python implementation of Proximity-based Multi-turn Optimization (ProxMO), a lightweight framework for training LLM agents in multi-turn reinforcement learning. It tackles context-dependent credit assignment in long-horizon tasks by modulating gradients based on episode success rates and aggregating step-level baselines via semantic proximity, plugging directly into GRPO pipelines. Users get boosted success rates on benchmarks like ALFWorld and WebShop through simple bash scripts for training.

Why is it gaining traction?

It delivers tangible gains—up to +28.9% on ALFWorld with 1.5B models and beats GPT-4o on key metrics—while adding just 1.09% overhead via vectorized operations, no extra networks required. Hyperparameter robustness across scales (1.5B to 7B) and plug-and-play setup make it a low-risk upgrade over group-based methods like GiGPO. Developers hook on the full ablation studies and production-ready efficiency for real multi-turn agent workflows.

Who should use this?

RL engineers tuning LLMs for embodied AI tasks in ALFWorld or WebShop, where sparse rewards kill standard RL. Researchers iterating on multi-turn optimization for text-based or hybrid environments needing better credit assignment without retraining from scratch.

Verdict

Early but promising official ProxMO implementation (36 stars, 1.0% credibility)—solid README, env setup scripts, and results make it worth a spin for LLM-RL experiments despite low maturity. Fork and benchmark against GRPO if proximity-based tweaks fit your multi-turn pipeline.

(198 words)