shawn0728 / OpenSearch-VL

What is OpenSearch-VL?

OpenSearch-VL provides a fully open Python recipe for training agentic multimodal deep search agents through high-quality data curation, diverse visual and search tools, and fatal-aware reinforcement learning. It lets you build agents that inspect blurry images, crop regions, run web searches, and chain tools for grounded answers on tough visual QA tasks—reproducing proprietary-level performance end-to-end with open data, models, and code. Users get SFT/RL pipelines plus ready inference scripts for benchmarks like VDR and InfoSeek.

Why is it gaining traction?

It stands out by closing the gap on closed training recipes: agentic cold-start SFT on 36k trajectories, fatal-aware GRPO RL that masks cascading tool failures without nuking good reasoning, and a shared toolset (crop, search, enhance) across stages. Developers dig the 10+ point benchmark lifts over baselines at 30B scale, plus plug-and-play eval with GPT-4o judging—no more guessing black-box agent behaviors.

Who should use this?

ML engineers fine-tuning vision-language models for tool-using agents in deep research QA, like identifying landmarks from photos then fact-checking via search. Researchers replicating agentic RL on multimodal data, or teams building open alternatives to proprietary search VLMs for knowledge-heavy apps.

Verdict

Grab it if agentic multimodal learning fits your stack—thorough docs and HF models make prototyping fast, despite 47 stars and 1.0% credibility signaling early days. Fork and contribute; it's a credible open bet on fatal-aware agents.

(198 words)