Luce-Org

Luce-Org / luce-megakernel

Public

Megakernel to match Apple Silicon Efficiency at 2x the Throughput on a RTX 3090

48
8
100% credibility
Found Apr 09, 2026 at 48 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Cuda
AI Summary

This project delivers a custom optimizer that runs a specific small hybrid AI language model at exceptional speed and energy efficiency on everyday NVIDIA graphics cards.

How It Works

1
📰 Discover Fast AI Trick

You hear about a clever way to make AI language models run much faster and more efficiently on your home computer's graphics card.

2
💾 Grab the Files

You download the simple package to your computer from the project page.

3
🛠️ Set It Up Quickly

You prepare everything with an easy installation step so it's ready to go.

4
⚡ Launch Speed Test

You give it a starting message and watch it process and generate text at blazing speeds.

5
📊 See the Results

Your screen fills with impressive numbers showing how much faster and power-saving it is compared to regular ways.

🚀 Enjoy Super Fast AI

Now you can run your own AI conversations on your PC lightning-quick, matching top efficiency without fancy new hardware.

Sign up to see the full architecture

4 more

Sign Up Free

Star Growth

See how this repo grew from 48 to 48 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is luce-megakernel?

Luce-megakernel fuses the entire forward pass of Qwen 3.5-0.8B—a hybrid DeltaNet/Attention LLM—into a single CUDA dispatch for decode on NVIDIA GPUs. It delivers 413 tok/s decode and 37k tok/s prefill on an RTX 3090, hitting 1.87 tok/J efficiency to match Apple silicon while doubling throughput. Install via pip, load weights from Hugging Face, and benchmark with provided scripts for single-user local inference.

Why is it gaining traction?

It crushes llama.cpp (1.55x decode, 3.4x prefill) and PyTorch (3.8x overall) on the same RTX 3090 by slashing kernel launches, proving CUDA kernels can close the efficiency gap with Apple silicon. Power-limit your GPU to 220W for peak tok/J without speed loss. Developers dig the open benchmarks and Discord for tweaking hybrid model runs.

Who should use this?

GPU tinkerers running local LLMs on Ampere+ cards like RTX 3090, especially those experimenting with Qwen's DeltaNet layers. Ideal for solo researchers benchmarking throughput vs. efficiency or porting hybrid models to consumer hardware. Skip if you need batching, quantization, or multi-model support.

Verdict

Grab it for proof-of-concept speedups on Qwen 3.5-0.8B—it's a solid CUDA template despite 48 stars and 1.0% credibility score signaling early-stage maturity. Production? Wait for generalization; for now, it's a sharp lesson in megakernel efficiency.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.