bethgelab

bethgelab / delta-belief-rl

Public

Official implementation of the ΔBelief-RL method.

bethgelab.github.iodelta-belief-rl
24
2
100% credibility
Found Feb 17, 2026 at 19 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

A research codebase for training compact AI agents in long-horizon guessing games using self-confidence updates as rewards, achieving superior performance to massive models.

How It Works

1
🔍 Discover smarter game-playing agents

You stumble upon a clever way to train small AI thinkers to guess secrets in games like 20 Questions by rewarding their growing confidence.

2
📥 Grab the ready toolkit

Download the simple bundle of tools that makes it easy for anyone to try this out.

3
🧠 Choose your starting thinker

Pick a small ready-made AI brain to build upon, like a quick learner sized just right.

4
🚀 Start the learning adventure

Hit go and let it play guessing games, improving by tracking its own hunches over many rounds.

5
🎯 See confidence magic happen

Watch in awe as your agent asks fewer questions and nails the secret faster than huge rivals.

6
🧪 Test on fresh puzzles

Challenge it with new games like city guessing or mysteries to prove its smarts carry over.

🏆 Celebrate the win

Enjoy your tiny agent's victory over giants, ready to explore even tougher interactive quests.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 19 to 24 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is delta-belief-rl?

Delta-belief-rl is the official GitHub repository for ΔBelief-RL, a Python method to train small LLMs as "Curious Information-seeking Agents" (CIA) in long-horizon games like 20 Questions. It uses changes in the model's own token probabilities (belief updates) as dense intrinsic rewards, skipping critics or process models. Run training via SLURM scripts on Qwen 1.7B/4B models, eval on OOD tasks like Guess My City or Customer Service—all with Ray, FSDP, and vLLM.

Why is it gaining traction?

It crushes baselines: 1.7B CIAs hit 24.8% success on 20 Questions (vs. GRPO's 16.5%), beating DeepSeek-V3.2 (670B) despite 98% fewer params, plus strong OOD generalization and scaling beyond training horizons. No reward engineering needed—just self-belief diffs for efficient exploration, fewer repeats, shorter trajectories. Devs dig the plug-and-play scripts and Hugging Face model exports.

Who should use this?

RLHF engineers building interactive LLM agents for diagnostics, customer support chats, or mystery-solving bots. Ideal for researchers prototyping intrinsic motivation without custom reward models, especially on multi-GPU clusters with SLURM.

Verdict

Promising official implementation from Tübingen AI Center, but 19 stars and 1.0% credibility signal early research code—docs are README-focused, no broad tests. Fork and validate on your setup before scaling.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.