voipmonitor

voipmonitor / rtx6kpro

Public

RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink

38
1
100% credibility
Found Mar 13, 2026 at 37 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

Community knowledge base with guides, benchmarks, and optimization tips for running massive language models on clusters of NVIDIA RTX 6000 Pro GPUs via PCIe without NVLink.

How It Works

1
🔍 Find the GPU Guide

You hear about this community notebook full of tips for squeezing top speed out of big AI thinkers on powerful graphics cards.

2
📖 Pick Your AI Model

Browse easy pages listing huge models like Qwen or GLM, noting which need 2, 4, or 8 cards for best results.

3
🖥️ Plan Your Card Setup

Read friendly advice on linking cards through computer slots for smooth teamwork without extra cables.

4
🚀 Apply Speed Boosts

Try proven tweaks like magic number settings that suddenly make everything 50% faster and feel super smooth.

5
📊 Measure Your Wins

Run simple tests to clock words flying out at hundreds per second, comparing to community scores.

✅ Power Up Huge AIs

Enjoy lightning-fast AI chats on your setup, joining others to share setups and celebrate breakthroughs.

Sign up to see the full architecture

4 more

Sign Up Free

Star Growth

See how this repo grew from 37 to 38 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is rtx6kpro?

RTX6kpro is a community wiki and benchmark suite for running massive LLMs like Qwen3.5-397B and GLM-5 on NVIDIA RTX 6000 Pro Blackwell GPUs in 2x to 8x PCIe setups without NVLink. Built in Python, it delivers configs for SGLang and vLLM, quantization guides for NVFP4/AWQ, and scripts to benchmark throughput and KLD quality on rtx 6000 series hardware. Developers get reproduction-ready Docker commands and PCIe topology tips to hit 100+ tok/s decode on 8x rigs.

Why is it gaining traction?

Unlike generic LLM repos, rtx6kpro focuses on rtx 6000 pro blackwell realities—PCIe bandwidth hacks, NCCL fixes for AMD EPYC, and MTP=2 sweet spots that boost throughput 50-70%. Benchmarks pit AWQ-INT4 vs NVFP4 with real numbers (e.g., 1662 tok/s at C=64), plus KLD eval pipelines to verify quality. It's the go-to for rtx 6000 vs 5090 debates in chat rtx github circles.

Who should use this?

AI infra engineers scaling inference on rtx 6000 ada or blackwell without NVLink, especially for MoE models like GLM-5 on 4x/8x PCIe servers. Teams testing github rtx test setups on Turin/Genoa hosts or optimizing SGLang for long-context Kimi-K2.5.

Verdict

Grab it if you're on rtx 6000 pro hardware—docs and benchmarks are solid despite 31 stars and 1.0% credibility score signaling early maturity. Low activity means verify community PRs, but it's a practical starting point over scattered Discord threads.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.