GuidedVLA

GuidedVLA / GuidedVLA

Public

[RSS 2026] GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization

guidedvla.github.ioproject_page
44
0
89% credibility
Found May 19, 2026 at 44 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

GuidedVLA is a robotics AI research project that improves how robots learn to perform household and manipulation tasks. Rather than treating a robot's decision-making as one monolithic system, it divides the robot's attention into specialized components: one focuses on relevant objects in the camera view, another understands which phase of a task the robot is in, and a third provides 3D spatial awareness. This modular approach helps robots generalize better to new situations—like different kitchens, lighting conditions, or object arrangements—without needing to relearn everything from scratch. The project builds on Physical Intelligence's openpi (π₀) foundation and provides both training pipelines and evaluation tools for robotics researchers.

How It Works

1
🔬 You discover a new robot learning approach

You hear about GuidedVLA, a research project that helps robots learn household tasks more reliably by focusing their attention on what matters most.

2
📚 You learn how the robot's brain is organized

Instead of one big brain, GuidedVLA divides the robot's thinking into specialized parts: one for finding objects, one for understanding task phases, and one for sensing depth.

3
🎯 You see the robot generalize better

The key benefit: robots trained this way adapt to new situations—like different kitchens or lighting—much better than before.

4
You choose your path
🏋️
Train a new robot policy

You collect robot demonstration data, configure what specialized heads to use, and let the training process teach the robot new skills

🧪
Test an existing robot policy

You download a pre-trained checkpoint and run it in a robot simulator to see how well it performs different tasks

5
🤖 Your robot takes action

The trained robot watches camera feeds, understands what you want it to do from your instructions, and generates smooth motor commands to complete tasks.

🎉 Your robot succeeds in new situations

Thanks to the specialized attention heads, your robot handles variations in objects, lighting, and scenes that it never saw during training.

Sign up to see the full architecture

4 more

Sign Up Free

Star Growth

See how this repo grew from 44 to 44 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is GuidedVLA?

GuidedVLA is a robotics foundation model that extends Physical Intelligence's π₀ framework with plug-and-play attention heads that specialize in different task-relevant factors. Instead of relying on end-to-end training to implicitly learn object grounding, spatial geometry, and temporal skill logic, GuidedVLA explicitly supervises dedicated attention heads using auxiliary signals. The system uses a ControlNet-style adapter that starts with zero contribution and gradually learns to inject factor-specific biases, preserving pretrained capabilities throughout training. Built in Python with full PyTorch support alongside the original JAX implementation, it ships with training scripts, evaluation pipelines for LIBERO-Plus and RoboTwin 2.0, and pretrained checkpoints on HuggingFace.

Why is it gaining traction?

The numbers are compelling. On LIBERO-Plus, GuidedVLA pushes total performance from 68.2% to 75.4% compared to the base π₀ model, with particularly strong gains on spatial (84.0%) and goal-oriented tasks (70.8%). The real-world results on ALOHA and PSI-Bot hardware show even larger gaps—in-domain generalization jumps from 55.8% to 75.8%. The modular design is the real hook: you can enable or disable object, depth, and skill heads independently, making it easy to experiment with which auxiliary signals matter for your specific task. The zero-initialized projection approach means you can add these specialized heads to an existing pretrained model without catastrophic forgetting.

Who should use this?

Robotics researchers working on manipulation tasks who need better generalization across object types, scene configurations, and lighting conditions. Developers fine-tuning VLAs on custom robot platforms will benefit most from the plug-and-play architecture—if your dataset includes object masks or skill labels, you can enable those heads with minimal configuration changes. The released checkpoints and datasets lower the barrier for anyone wanting to reproduce results or build on this work. If you're evaluating VLAs for deployment in household or lab environments with significant variation, this is worth benchmarking against simpler approaches.

Verdict

GuidedVLA delivers measurable improvements on manipulation benchmarks with a clean, modular architecture that balances specialization against backbone preservation. The 0.9% credibility score reflects its academic origins and early release stage—44 stars and a recent RSS submission mean you're adopting cutting-edge research, not production-tested infrastructure. Documentation is thorough and the codebase includes Docker support and multi-GPU training, but test coverage and community tooling are still maturing. Worth exploring for research purposes or as a strong baseline for manipulation tasks, though teams needing battle-tested deployment solutions may want to wait for broader validation.

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.