0xClandestine

0xClandestine / mirror-sd

Public

DFlash block-diffusion speculative decoding running on Apple Silicon via MLX, with an ANE execution path that explores heterogeneous accelerator dispatch.

49
1
100% credibility
Found Apr 16, 2026 at 49 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

This project speeds up AI language generation on Apple computers using clever prediction tricks.

How It Works

1
🖥️ Discover faster AI chats

You hear about Mirror-SD, a fun way to make AI conversations zoom on your Apple Mac.

2
📥 Set it up simply

Follow easy steps to download and prepare everything on your computer.

3
🤖 Pick your AI helpers

Choose a main smart thinker model and a speedy sidekick to boost it.

4
✨ Ask and watch magic

Type a question or story idea, and see responses appear lightning fast.

5
🌐 Start endless chats

Launch a web chat room to talk with your supercharged AI anytime.

🚀 Speedy creations unlocked

Enjoy quicker answers for writing, learning, or fun, all on your Mac.

Sign up to see the full architecture

4 more

Sign Up Free

Star Growth

See how this repo grew from 49 to 49 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is mirror-sd?

Mirror-sd brings DFlash block diffusion for flash speculative decoding to Apple Silicon via MLX, accelerating LLM inference by using a compact draft model to predict token blocks in parallel. Developers get 1.2-2.2x speedups on Qwen3.5-27B (M4 Max) and 3.55x on Qwen3-8B, with an experimental ANE execution path for heterogeneous accelerator dispatch between GPU target and ANE draft. Install via pip, run OpenAI-compatible servers, CLI generation/benchmarks, or Python API for custom loops.

Why is it gaining traction?

It outperforms baselines in llama-benchy at all context depths, with fixed block sizes beating adaptive methods like KOD on these models. Prompt caching, streaming endpoints, and easy HF draft model loading make it plug-and-play for tools like llama-benchy. The ANE path hints at future sd mirror optimizations via Apple accelerators.

Who should use this?

Apple Silicon users running Qwen3/Qwen3.5 locally for chat, code gen, or benchmarks. ML engineers evaluating DFlash speculative decoding with block diffusion speedups. Devs needing an OpenAI server drop-in for speculative inference without CUDA.

Verdict

Worth trying for Apple devs chasing inference gains—benchmarks deliver, docs guide setup, CLI/server work out-of-box. At 49 stars and 1.0% credibility, it's early but MIT-licensed and focused; test on your hardware before production.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.