Fediory

Fediory / Grid-Sampler

Public

[ICML 2026] Official Implementation of "See What Matters: Differentiable Grid Sample Pruning for Generalizable Vision-Language-Action Model"

fediory.github.ioGrid-Sampler
47
2
100% credibility
Found May 19, 2026 at 48 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

Grid Sampler is an academic research project (accepted to ICML 2026) that makes robot vision systems faster and more efficient. Instead of processing every pixel in a camera image, the system learns to automatically focus on the most important visual regions - similar to how humans naturally focus on relevant objects rather than examining every detail. The project integrates with established robot learning frameworks (LeRobot and openpi) and provides tools for recording robot demonstrations, training policies in simulation or on real hardware, and deploying capable robots that can perform manipulation tasks like picking and stacking objects. It includes support for various robot types and camera systems, with pre-trained models available for common tasks.

How It Works

1
🔬 You discover a smarter way to teach robots to see

A researcher shares Grid Sampler - a breakthrough that helps robots focus on what matters in camera images, using less computer power.

2
🎯 Your robot learns to focus on what matters

Instead of examining every pixel, the system learns which parts of an image are important - like how you focus on a coffee cup instead of the entire table.

3
📹 You connect your robot's cameras

Set up your robot arm with cameras, configure how it sees the world, and prepare to record demonstrations of tasks.

4
🎮 You teach the robot by showing it what to do

Use a controller to move the robot arm while the cameras record. The robot watches and learns from your movements.

5
Choose how to train your robot
🎮
Practice in simulation first

Test your robot's behavior in a virtual environment where mistakes don't matter.

🦾
Train directly on the real robot

Let the robot learn from your real-world demonstrations.

6
🚀 Your robot gets faster at understanding

Grid Sampler compresses what the robot sees into only the important parts, making decisions quicker and using less energy.

🎉 Your robot completes tasks on its own

After training, the robot can pick up objects, stack items, and handle new situations it hasn't seen before - all using its smart, focused vision.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 48 to 47 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is Grid-Sampler?

Grid-Sampler is a research implementation from ICML 2026 that prunes visual tokens in robot policies. Instead of feeding every pixel through a vision encoder, it learns to sample only the most informative regions of an image. The system predicts K normalized coordinates, uses bilinear sampling to extract features at those locations, and optionally injects coordinate embeddings back into the tokens. It ships as a JAX implementation (built on openpi/pi0) and a PyTorch fork of LeRobot, supporting policies like SmolVLA, ACT, and Diffusion.

Why is it gaining traction?

The hook is generalization: dense visual representations struggle when robots encounter new objects or viewpoints. By pruning to a sparse set of learnable tokens, Grid-Sampler forces the policy to focus on task-relevant regions rather than memorizing pixel patterns. The approach is framework-agnostic, working with both JAX and PyTorch stacks, and integrates into existing pipelines through simple config flags like `use_grid_token_sampler=true`. Pre-trained checkpoints are available on Hugging Face.

Who should use this?

Robotics researchers working on imitation learning who want to improve out-of-distribution generalization. If you're fine-tuning pi0 or SmolVLA on a new task and noticing brittle behavior on novel objects, this is worth benchmarking. Simulation-only workflows (LIBERO, ALOHA) are well-supported; real-world deployment requires more setup. Not for production robotics teams yet.

Verdict

The idea is solid and the paper is peer-reviewed, but the repo has 47 stars, minimal test coverage, and a credibility score of 1.0%. Treat this as a research prototype to evaluate on your specific task, not a drop-in library. Check the openpi and LeRobot subdirectories for working examples, and verify the JAX-to-PyTorch weight conversion works for your policy before committing.

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.