xgbj

xgbj / sparse-mask-attention

Public

高性能短序列稀疏Mask Attention CUDA算子，针对<1K序列+75%稀疏度优化

45
3
100% credibility
Found Mar 24, 2026 at 45 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

This repository offers a finely tuned computation engine for efficient attention processing in AI models, specialized for short sequences with mostly ignored positions.

How It Works

1
🔍 Discover the Speed Tool

You find this handy kit on a code-sharing site that promises to make AI pattern-matching way faster for short, spotty data.

2
📦 Set It Up

You grab the pieces and add them to your AI workspace with a simple copy-paste command, and it builds itself ready to go.

3
📊 Prepare Your Pieces

You gather your question data, memory data, value data, and a simple yes/no pattern map showing which parts to focus on.

4
✨ Run the Fast Match

With one easy call, it crunches the patterns super quickly, giving you the focused results in a flash.

5
🧪 Check It Works

You compare the outputs to the usual slow way and see they match perfectly, so you know it's spot on.

6
⏱️ Measure the Boost

You time it against other tools and grin at the huge speedup – up to 35 times quicker on your setup.

🚀 AI Runs Blazing Fast

Now your AI model thinks lightning-quick on sparse short patterns, saving time and powering better results.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 45 to 45 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is sparse-mask-attention?

This Python project delivers a CUDA kernel for sparse attention with bool masks on short sequences under 1K tokens and 75% sparsity, slashing latency from 16ms to 0.5ms on RTX 3080. It takes Q/K/V tensors and a [B,H,N,N] attention mask—think attention mask explained for BERT or mask attention pytorch—and outputs the attended result via a simple `sparse_attention(q, k, v, mask)` call. Developers get drop-in speed for sparse attention mask scenarios without rewriting models.

Why is it gaining traction?

It crushes baselines: 1.6x faster than Triton, 1.9x over cuDNN SDPA, and 2.3x ahead of FlashInfer at 27 TFLOPS, all while handling custom sparse masks that dense kernels like flash-attn skip. The hook is `run.py` for instant correctness checks, perf scaling, and head-to-head compares, plus bit-packed masks cutting memory 8x. For mask attention transformer tweaks or trainable dynamic mask sparse attention, it's a quick win over generic PyTorch refs.

Who should use this?

ML engineers fine-tuning transformers with sparse patterns, like mask attention in BERT, mask DINO GitHub models, or 3D instance seg with mask attention free transformer. Vision devs dealing with attention_mask img shape in ComfyUI or mask R-CNN GitHub flows. Anyone optimizing short-seq inference where github mask output or input mask sparsity matters.

Verdict

Grab it if sparse attention mask perf is your bottleneck—docs, benchmarks, and API are solid for eval. Low 1.0% credibility from 45 stars means watch for maturity, but MIT license and CUDA setup make prototyping low-risk.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.