noonghunna

noonghunna / qwen36-dual-3090

Public

Qwen3.6-27B on dual RTX 3090 — TP=2 recipe, vLLM nightly, MTP + fp8 KV, validated for concurrent serving

43
3
100% credibility
Found May 02, 2026 at 43 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Shell
AI Summary

Set of ready-to-launch configurations to run a large AI language model on dual RTX 3090 graphics cards for fast local serving with full features like long context, vision, and tools.

How It Works

1
🔍 Find the guide

You search online for ways to run a powerful AI helper on your home computer with two special graphics cards.

2
✅ Check your setup

Make sure you have two RTX 3090 graphics cards, enough storage, and the right software basics like Docker.

3
📥 Download everything

Run a simple setup command to grab the AI model and preparation files safely onto your computer.

4
🚀 Launch your AI

Start the server with one easy command and watch it boot up, using both graphics cards together for speed.

5
Pick your style
⚖️
Balanced

Great all-around speed with pictures and long talks.

⚡
Turbo

More chats happening together without slowing down.

💻
Code boost

Lightning replies especially for writing programs.

6
💬 Start chatting

Send questions through a simple web link, like asking about France's capital, and get smart answers instantly.

🎉 Your private AI is ready

Now you have a fast, smart assistant at home that handles long stories, images, tools, and multiple users – all private and free from cloud costs.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 43 to 43 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is qwen36-dual-3090?

This shell recipe deploys Qwen3.6-27B on dual RTX 3090 GPUs via vLLM nightly and TP=2, delivering a local OpenAI-compatible API at http://localhost:8010. Users get full 262K context with vision, tools, streaming, and concurrent serving up to 4 streams, plus variants like fp8 MTP for balance or DFlash for code speed. It solves cramming a multimodal 27B model onto consumer hardware without dropping features or throughput.

Why is it gaining traction?

Validated benchmarks show 71-89 TPS single-stream and 257 TPS aggregate at 4 concurrent, beating single-GPU limits on long contexts. Docker Compose files swap effortlessly between general, turbo (9x KV pool), and DFlash setups, with setup.sh handling model downloads and verification. Stays fresh on vLLM nightly while patching just enough for stability.

Who should use this?

AI devs with dual 3090s running team inference, RAG chains, or multi-agent workflows needing 2-4 concurrent requests at 262K ctx. Code generators chasing 128 TPS peaks via DFlash. Local serving enthusiasts ditching cloud for validated, full-featured Qwen36 performance.

Verdict

Grab this if you have dual RTX 3090s—it's a battle-tested recipe with scripts, benchmarks, and verify checks for quick wins. 1.0% credibility from 43 stars flags its niche maturity, but docs and validation make it dependable; active forks live in club-3090 now.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.