bstnxbt

bstnxbt / dflash-mlx

Public

Lossless DFlash speculative decoding for MLX on Apple Silicon

402
24
69% credibility
Found Apr 15, 2026 at 424 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

A tool that speeds up AI text generation on Apple Silicon computers for faster and more efficient responses.

How It Works

1
🔍 Discover Speedy AI

You hear about a simple tool that makes AI conversations on your Apple computer much faster and smoother.

2
📥 Set It Up

With one easy download, you add the speed booster to your computer—no complicated steps needed.

3
🧠 Pick Your AI Friend

Choose a smart AI companion from a list of ready-to-use options that work perfectly with the tool.

4
Start Chatting
⚡
Quick Answer

Get an instant response to your question right away.

🌐
Web Chat

Launch a chat window on the web for ongoing talks.

5
🚀 Watch the Magic

Your AI thinks and replies super fast, like lightning, giving you smooth and accurate results every time.

🎉 Faster Chats Unlocked

Now you enjoy quicker, smarter AI help for writing, ideas, or fun conversations without waiting.

Sign up to see the full architecture

4 more

Sign Up Free

Star Growth

See how this repo grew from 424 to 402 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is dflash-mlx?

DFlash-MLX delivers lossless speculative decoding for LLMs on Apple Silicon using Python and the MLX framework. A compact draft model generates up to 16 tokens in one parallel pass, verified instantly by the target model for exact greedy output—no quality loss, just speed. Install via pip, generate with `dflash --model Qwen/Qwen3.5-9B --prompt "your text"`, or serve OpenAI-compatible endpoints with `dflash-serve`.

Why is it gaining traction?

It crushes baselines with 2-4x speedups on Qwen3.5 models (e.g., 197 tok/s vs 53 on M5 Max), auto-resolving drafts from Hugging Face. Streaming, chat templates, tools, and benchmarks via `dflash-benchmark` make it seamless over plain MLX. Echoes github lossless scaling vibes—pure throughput without artifacts.

Who should use this?

ML engineers on Apple Silicon running local Qwen inference for chat apps, RAG pipelines, or dev tools like Continue/Open WebUI. Anyone benchmarking tok/s on M-series chips tired of slow AR decoding.

Verdict

Grab it for MLX Qwen workflows—402 stars, polished README/benchmarks, MIT license signal reliability despite 0.7% credibility score. Early but production-ready; scales to 16k ctx with env tweaks.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.