pUmpKin-Co / ComplementaryRL

What is ComplementaryRL?

ComplementaryRL is a Python framework for training experience-driven RL agents, where policy actors and experience extractors co-evolve to make learning more efficient. It solves the problem of agents wasting compute on redundant experiences by building a centralized memory bank that retrieves, updates, and distills useful trajectories on the fly. Developers get plug-and-play integration into agent pipelines, with quick starts for baselines and full co-training via simple config tweaks.

Why is it gaining traction?

It stands out by decoupling actor rollouts from experience processing in an async loop, boosting sample efficiency without slowing training—key for scaling LLM-based agents. Features like diversified retrieval, actor-critic reflection on memories, and subgroup advantages prevent bias, delivering measurable gains over vanilla RL. Built on battle-tested stacks like vLLM and Ray, it hooks devs chasing state-of-the-art agent performance with minimal setup.

Who should use this?

RL engineers building LLM agents for games like MiniHack or tool-use environments, especially those already on the ROLL framework. Ideal for researchers iterating on experience replay in multi-turn tasks, or teams optimizing policy training under compute constraints.

Verdict

Worth a spin for agent RL experiments—solid paper, clear examples, and extensible configs make it approachable despite 19 stars and 1.0% credibility score. Still early; expect some ramp-up on docs and tests before production.

(198 words)