zoharbabin

zoharbabin / web-researcher-mcp

Public

MCP server (Go) for AI assistants: web search, content extraction, academic/patent/news research. Multi-provider routing, 4-tier scraping, search lenses. Works with Claude, Cursor, and any MCP client.

zoharbabin.github.ioweb-researcher-mcp ai ai-agent ai-tools brave-search claude
10
1
85% credibility
Found May 23, 2026 at 10 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Go
AI Summary

web-researcher-mcp is a production-ready tool that gives AI assistants the ability to search the internet, read web pages, extract content from documents and videos, and conduct thorough multi-step research. It works by connecting to your AI assistant through a standard protocol and providing multiple specialized tools: quick web searches, image searches, news lookups, academic paper searches, patent research, and full-content extraction from any URL. The system automatically handles technical details like finding alternative search providers if one is slow, protecting against common web security risks, and organizing research sessions. It can be used simply for fast answers or configured for deep investigative research across many sources. The project is open-source, well-documented, and maintained with comprehensive tests.

How It Works

1
🤖 Your AI assistant gains research superpowers

You connect the web researcher to your favorite AI assistant like Claude, and it suddenly becomes able to search the internet and read any webpage, PDF, or video.

2
🔍 You ask your AI about anything current

Instead of being stuck with outdated knowledge, you can now ask 'What's the latest news on quantum computing?' or 'Find me recent research on this topic' and get real answers from the live web.

3
You choose how deep to go
⚡
Quick search

Ask a question, get URLs and summaries back in seconds

📚
Deep research

Trace connections across multiple sources, extract full articles, and track your investigation step by step

4
🔄 The system automatically finds the best sources

If one search engine is slow or unavailable, it quietly switches to another so you always get results. You never have to worry about technical failures.

5
🛡️ Everything stays safe and private

The system protects against common web threats, won't accidentally access private networks, and keeps your research sessions organized and separate.

✨ You get comprehensive, cited research

Instead of generic answers, you receive well-researched information with clear citations, pulled from the most relevant and authoritative sources on the web.

Sign up to see the full architecture

4 more

Sign Up Free

Star Growth

See how this repo grew from 10 to 10 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is web-researcher-mcp?

An MCP server written in Go that gives AI assistants the ability to search the web, extract content from any URL, and conduct multi-source research. It connects to Claude Code, Cursor, or any MCP-compatible client and exposes a suite of research tools: web search, page scraping, academic paper search, patent lookup, news search, and image search. The server routes queries across multiple search providers (Google, Brave, Serper, SearXNG, SearchAPI.io) with automatic fallback if one fails. Content extraction uses a 4-tier pipeline that tries lightweight methods first and escalates to headless browser rendering only when needed.

Why is it gaining traction?

The MCP ecosystem is exploding, and this fills a real gap: AI assistants that can reason about the world but not access it. The multi-provider routing is the hook -- configure `SEARCH_ROUTING=brave,google,serper` and the server handles failover automatically. Search lenses let you focus results on curated domain lists (programming, legal, medical) without manual site filtering. For teams running shared MCP deployments, the enterprise features matter: OAuth 2.1 authentication, per-tenant rate limiting, audit logging, and SSRF protection. The 4-tier scraper handles everything from simple HTML to JavaScript-heavy SPAs without requiring separate infrastructure.

Who should use this?

Developers building AI-assisted workflows that need live web data: automated competitive analysis, research pipelines, fact-checking agents, or documentation tools that pull from changing sources. Researchers will appreciate the academic and patent search tools. Teams running shared Claude or Cursor instances in enterprise environments benefit from the security controls. Solo developers can use Brave Search for free high-volume searching without Google API key setup.

Verdict

This is a serious, well-architected project for a niche that's heating up fast. The 0.8500000238418579% credibility score reflects early-stage maturity -- only 10 stars means limited community validation, though the comprehensive documentation and test coverage suggest the author knows what they're doing. If you're evaluating MCP servers for web research, this is worth a weekend proof-of-concept. The multi-provider routing and search lenses alone justify the setup effort over rolling your own. Just budget time for API key configuration and understand that the free-tier Google limits (100 queries/day) will require a paid upgrade for anything beyond experimentation.

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.