moiseshorta / CoDiCodec-Flow

What is CoDiCodec-Flow?

CoDiCodec-Flow is a Python project that generates audio in real-time using flow matching, a modern take on diffusion models. You feed it a short audio clip, and it produces an arbitrarily long continuation that matches the style and feel of your prompt. The magic happens in a compressed latent space created by CoDiCodec, which shrinks 48kHz stereo audio by 128x while preserving musical structure. The model trains on these latents using Conditional Flow Matching with a block-causal transformer architecture that respects chunk boundaries and supports efficient streaming inference. Out of the box, you get a CLI for preprocessing audio, training models, and generating samples, plus a realtime mode that streams audio directly to your speakers with live keyboard controls for adjusting temperature, context length, and solver settings.

Why is it gaining traction?

The Apple Silicon support is the headline feature here. This runs in real-time on a 36GB M-series MacBook without a GPU, which is genuinely rare for audio generation models. The block-causal design means you can generate unlimited-length audio chunk-by-chunk without waiting for full-sequence inference. The realtime module includes surprisingly polished controls: adjust diffusion steps, solver type, temperature, and context window on the fly with keyboard input. The TUI training monitor is a nice touch, showing progress bars and loss metrics directly in the terminal. For a niche audio generation task, the documentation is thorough and the example audio demonstrates clear improvement across millions of training steps.

Who should use this?

Musicians and producers exploring AI-assisted continuation or improvisation will find this most useful. Researchers studying flow matching for audio will appreciate the clean architecture and MPS compatibility for rapid experimentation. Developers building realtime audio applications like AI accompanists or interactive music systems have the most to gain, though they should expect to invest significant time in training and tuning. If you need a production voice clone or text-to-speech system, look elsewhere—this is optimized for musical continuation, not speech.

Verdict

This is a serious research project from a credible author with solid fundamentals, but the 17 stars and early-stage maturity mean you're signing up for some rough edges. The architecture is well-documented, the Apple Silicon support works, and the realtime streaming mode is genuinely impressive. Start with the smoke test and example audio before committing to training.