sirohikartik / tinygpt

What is tinygpt?

TinyGPT delivers a custom C++ inference engine for a 30M parameter GPT model trained on the TinyStories dataset from GitHub. It runs autoregressive text generation on macOS CPUs, handling prompts up to 128 tokens with GPT-2 compatible tokenization and features like temperature sampling. Build once with make, then execute ./a.out for instant story-like outputs from inputs like "Hello, how are you?".

Why is it gaining traction?

This tinystories version stands out with SIMD-optimized CPU speed on Apple Silicon, no Python runtime or heavy frameworks needed—pure C++ for tinygpt huggingface-style models. Developers hook into its modular generation tweaks, like max tokens and context cropping, making it a lightweight alternative to bloated inference setups for small backbones akin to tinygpt-v.

Who should use this?

ML tinkerers porting TinyStories models to edge devices, C++ enthusiasts rebuilding GPT inference from weights in .npy format, or educators demoing transformer generation without GPUs. Perfect for macOS users prototyping efficient multimodal large language model via small backbones like tinygpt-v huggingface variants.

Verdict

At 16 stars and 1.0% credibility score, it's immature with sparse tests and Linux/Windows tweaks needed, but docs guide quick setup. Grab it for educational custom engine experiments; pass for anything reliable.

(178 words)