zakirkun

zakirkun / wayang.rs

Public

Lightweight fast LLM.

16
1
100% credibility
Found Apr 16, 2026 at 16 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Rust
AI Summary

wayang.rs is a Rust-based engine for running large language models on everyday CPUs using GGUF files, featuring a simple chat tool and a web server compatible with standard AI chat apps.

How It Works

1
🔍 Discover wayang.rs

You find wayang.rs, a friendly tool that lets you chat with smart AI right on your own computer without needing fancy hardware.

2
📥 Set it up

Download the program and get it ready on your computer in a few simple steps.

3
🧠 Grab an AI model

Pick a ready-made AI brain file and bring it to your computer.

4
Choose your way to chat
🗣️
Quick chat

Start typing messages and get instant smart replies.

🌐
Web sharing

Turn it into a web chat anyone can use from their browser.

5
✨ Magic happens

Watch the AI think and respond just like a helpful friend, fast and private.

😊 Your AI companion

Now you have your own clever assistant for questions, ideas, or fun chats anytime.

Sign up to see the full architecture

4 more

Sign Up Free

Star Growth

See how this repo grew from 16 to 16 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is wayang.rs?

wayang.rs delivers CPU-only LLM inference in Rust, loading GGUF models from local files or Hugging Face for chat completions and text generation. Run one-shot prompts via CLI, benchmark tokens-per-second, or spin up an OpenAI-compatible HTTP server on port 11434 with SSE streaming and metrics. It's a lightweight fast LLM setup solving GPU-free local AI on laptops or servers.

Why is it gaining traction?

Zero GPU needed, with memory tricks like layer streaming (one block in RAM) and KV quantization slashing peak usage—think lightweight fast laptop for models. Prefix caching speeds repeated prompts, speculative decoding boosts throughput, and native CPU tuning via Makefile hits solid tok/s on consumer hardware. Devs dig the betfair lightweight github feel: drop-in server, no bloat.

Who should use this?

Rust backend devs embedding local inference in tools, like lightweight RAG github stacks or wayang cs agents. Suits solo hackers prototyping self-hosted chat APIs, edge apps on lightweight fast electric scooters (metaphorically), or teams dodging cloud LLM bills without sacrificing speed.

Verdict

Test it for lightweight LLM serving—1.0% credibility score matches 16 stars and v0.1 status, but strong docs, CLI, and benchmarks lower the risk. Mature enough for toys/small models; watch for prod stability.

(187 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.