real-stanford / dice-rl

What is dice-rl?

dice-rl lets you finetune diffusion and flow-based behavior cloning policies into full RL agents using distribution contractive iterations, turning expert priors into sample-efficient skill masters for robotics tasks. Start with Robomimic datasets from Hugging Face, pretrain BC policies via simple hydra commands like `python script/run.py --config-name=pre_flow_matching_mlp`, then finetune online with PPO or residual flow distillation on state or image observations. Built in Python atop MuJoCo and Robosuite, it handles manipulation benchmarks like D3IL avoiding/pushing/stacking and furniture assembly out of the box.

Why is it gaining traction?

It bridges prior probability shifts in offline RL by distilling generative policies stably, outperforming pure online RL on long-horizon tasks with fewer samples—key for dice rl settings where BC priors like deep image prior github or human body prior github need rl coaching. Developers grab it for plug-and-play HF checkpoints/datasets via `bash script/download_hf.sh`, quick evals with `python script/eval_rl_checkpoint.py`, and tunable configs balancing BC loss and RL updates. Low stars (19) but rising among prior fitted networks github fans seeking dice offline rl without prior project doo.

Who should use this?

Robotics PhD students or RL engineers finetuning vision-based manipulators on Robomimic/D3IL data, especially for sparse-reward stacking or furniture tasks. Ideal if you're prototyping dice rlc with image policies but tired of unstable online RL from scratch.

Verdict

Grab it for dice rl experiments on standard benches—docs guide setup/eval well, but 1.0% credibility and 19 stars signal early-stage code; test configs first before production. Strong start for prior products in manipulation RL.

(198 words)