Incept5 / gemma4-benchmark

What is gemma4-benchmark?

This Python tool runs apple mlx benchmarks on Gemma 4 and Qwen 3.5 vision-language models using mlx-vlm on Apple Silicon, testing prefill and decode throughput from 4k to 256k context lengths. It quantifies TurboQuant KV cache effects—2.5-bit compression that boosts decode speeds up to 19% on long contexts without extra memory. Users get interactive HTML charts, markdown tables, and raw JSON data after a quick pip install and model setup in their LM Studio directory.

Why is it gaining traction?

In the github mlx community, it stands out for mlx lm benchmarks tailored to M5 mlx benchmark hardware like Apple M5 Max, comparing 4bit/8bit/bf16 quants and TurboQuant vs standard runs—ideal for mlx vs gguf benchmark or mlx vs pytorch benchmark debates. The YAML config lets you tweak models, contexts, and sampling, with subprocess isolation for precise peak memory stats. Developers grab it for mlx github examples on long-context LLM perf, including mlx benchmark llm insights beyond short prompts.

Who should use this?

Apple ML engineers tuning Gemma or Qwen models for mlx github apple workflows, especially those hitting KV cache bottlenecks at 128k+ tokens. Github mlx lm enthusiasts validating TurboQuant in production, or researchers comparing mlx whisper benchmark styles to denser setups. Vision-language devs on M5 needing hard numbers before deploying to mlx github openai server or mlx github swift examples.

Verdict

Solid starter for mlx lm benchmark on Apple hardware—run your own tests in minutes with clean reports—but at 17 stars and 1.0% credibility score, it's early days; fork and contribute if you need broader model support or tests. Worth it for M5 owners eyeing Gemma 4 efficiency.

(198 words)