aim-uofa / GSI-Bench

What is GSI-Bench?

GSI-Bench is a Python-based gsi benchmark for exploring spatial intelligence from a generative perspective, accepted to CVPR2026. It tests how well generative models handle 3D spatial tasks in indoor scenes—like moving, rotating, scaling, or removing objects—via edited image outputs. Users get four key metrics: Instruction Compliance, Spatial Accuracy, Appearance Consistency, and Edit Locality, with ready-to-run evaluation scripts after downloading datasets and weights.

Why is it gaining traction?

This gsi technology benchmark stands out by automating 3D-aware evaluation for any generative model, skipping manual annotation hassles common in scene editing tests. Developers plug in their model's outputs (e.g., via simple folder structure and bash eval.sh), get JSON reports across real and synthetic datasets, and even optional MLLM-based scoring. Its transparency—full data pipelines for RoboTHOR and MesaTask scenes—builds trust for reproducible research.

Who should use this?

Computer vision researchers benchmarking generative models on spatial reasoning, especially indoor scene manipulation. Teams building image editors or 3D-aware diffusion models will value the plug-and-play eval for tasks like "move chair 20cm left." Anyone prepping CVPR-style papers on gsi needs its metrics for quick baselines.

Verdict

Try it for spatial intelligence evals—solid docs and tests make setup straightforward despite 18 stars and 1.0% credibility signaling early maturity. Worth watching as CVPR2026 drives adoption.

(198 words)