hpenedones

hpenedones / fastflowlm-docker

Public

Run LLMs on AMD Ryzen AI NPU (Linux)

12
2
100% credibility
Found Mar 11, 2026 at 10 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Dockerfile
AI Summary

A Docker-based setup that allows running large language models and speech-to-text on AMD Ryzen AI hardware neural processing units under Linux using FastFlowLM.

How It Works

1
🖥️ Discover Fast AI for Your AMD PC

You hear about an easy way to run smart AI language models super fast using the special AI chip built into your AMD Ryzen computer.

2
🔧 Get Your Computer Ready

Check that your Linux computer has the right updates, driver, and tools so the AI chip can be accessed safely.

3
📦 Prepare the AI Runner

Download the simple package and build it once to create your personal AI environment that talks to the chip.

4
📥 Pick and Download a Model

Choose a clever AI brain like Llama 3.2 and watch it download quickly to your setup for instant use.

5
Launch and Choose Your Way
💬
Chat Live

Jump into real-time conversations where the AI responds blazingly fast on your hardware.

🌐
Set Up a Service

Create a background service so apps and tools can talk to your AI anytime.

🎤
Enable Voice Features

Add speech-to-text to transcribe audio or chat by speaking to the AI.

⚡ Experience Lightning AI

Delight in generating 60+ words per second with full AI power running locally on your chip, no internet required.

Sign up to see the full architecture

4 more

Sign Up Free

Star Growth

See how this repo grew from 10 to 12 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is fastflowlm-docker?

Fastflow docker is a Dockerfile project that lets you run LLMs locally on Linux using AMD Ryzen AI NPUs, solving the lack of official support for tools like FastFlowLM. Pull models like Llama 3.2 1B or Qwen3, chat interactively via CLI, spin up an OpenAI-compatible API server on port 8000, or transcribe audio with Whisper—all accelerated purely on the NPU at 60+ tokens/s decode speeds. It's a drop-in container for how to run LLMs locally with Docker on XDNA hardware like Strix Point.

Why is it gaining traction?

It stands out by bridging the gap in Linux NPU support, delivering GPU-free inference with solid benchmarks (e.g., 88 tok/s prefill on Qwen3 0.6B) and easy commands like `docker run fastflowlm run llama3.2:1b` or `serve`. The 440MB image builds once and runs anywhere with the right kernel driver, plus Whisper integration for speech-to-text in chats or API calls. Developers dig the validate command for quick NPU checks and persistent model caching via volumes.

Who should use this?

Linux users with AMD Ryzen AI laptops (Strix Point, Kraken Point) experimenting with run LLMs locally on NPU for offline apps. AI tinkerers building local chatbots or voice tools needing fast, low-power inference without cloud dependency. Edge ML devs prototyping OpenAI API endpoints on battery-powered hardware.

Verdict

Grab it if you have compatible AMD gear—docs are thorough, usage is straightforward, and it works as advertised despite 10 stars and 1.0% credibility score signaling early maturity. Test on your setup first; it's niche but fills a real void until official Linux NPU stacks mature.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.