joshmorgan1000 / ane

What is ane?

ane provides bare-metal ARM SME2 assembly kernels mimicking Apple's Neural Engine on M4 chips, delivering ANE-matched performance for 153 single-threaded ML ops like elementwise arithmetic, activations, reductions, matmuls, convolutions, attention, losses, and optimizers. Developers include simple C++ headers and link a static lib via CMake for drop-in functions—think ane::kernel::matmul_fp32(A, B, C, M, N, K)—targeting armv9-a+sme2+sve2+sme-lutv2 hardware. It solves the black-box CoreML gap, offering direct throughput for custom inference without Apple's github containers or MDM wrappers.

Why is it gaining traction?

Unlike official apple github repositories or apple neural accelerators APIs, ane reverse-engineers ANE perf quirks—int8 favoritism, no FP16, mutual slowdown with CoreML—hooking devs chasing raw Apple Silicon speed. Flash attn, GQA decode, fused norms, and LUT ops stand out for transformer workloads, beating generic ARM libs in benchmarks while staying single-threaded and lightweight.

Who should use this?

ML engineers on M4 optimizing custom kernels for edge inference, like conv2d_bias_relu or sdp_attn_decode_cached. Transformer hackers prototyping rope or kv-cache appends without frameworks. Low-level perf tuners benchmarking against apple neural accelerators amid anemone-like experimental vibes.

Verdict

Intriguing for Apple Silicon tinkerers, but 19 stars and 1.0% credibility score flag it as raw prototype—not prod-ready, as warned. Pair with tests if you're aneurysma-kopf deep into bare-metal ML.

(198 words)