chu2bard

chu2bard / promptctl

Public

Prompt versioning and A/B testing framework

experimentation llm prompts testing typescript
11
0
100% credibility
Found Feb 11, 2026 at 11 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
TypeScript
AI Summary

promptctl is a tool for saving versions of AI prompts, comparing changes between them, setting up A/B tests, and viewing performance reports.

How It Works

1
🔍 Discover promptctl

You hear about a simple tool that helps save different versions of your AI instructions and test which ones work best.

2
🛠️ Set it up

You quickly install the tool on your computer so it's ready to use anytime.

3
🖋️ Save your instructions

You type or paste your AI instruction, give it a name, and save the first version safely.

4
🔍 Spot the changes

You pick two versions and instantly see exactly what's different between them, like a friendly side-by-side view.

5
🧪 Start a test

You set up a simple experiment to randomly show one version or the other to users and measure results.

6
📈 Gather real results

As people interact with your AI, the tool automatically tracks speed, length of responses, and quality scores.

🎉 Unlock the winner

You view a clear summary report that reveals which version performs better, helping you pick the best one moving forward.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 11 to 11 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is promptctl?

promptctl is a TypeScript CLI tool for versioning AI prompts—like those for ChatGPT, Claude, or Copilot—and running A/B tests on them. It stores versions in SQLite, letting you add prompts from stdin or files, diff changes between versions, create experiments with control and treatment splits, and track metrics like latency, token count, and quality scores. Developers get a lightweight way to iterate on github ai prompts without scattering them across repos.

Why is it gaining traction?

It stands out with dead-simple CLI commands for prompt versioning best practices: pipe in a prompt, diff v1 vs v2, spin up A/B experiments, and pull reports with deltas on perf metrics. Unlike bloated prompt versioning tools in Python or MLflow integrations, this open source TypeScript option skips heavy deps for quick local experiments—think prompt github copilot vscode tweaks or Cursor promptql tests. The colored diffs and summary stats hook devs chasing prompt versioning langchain or Langfuse vibes without the overhead.

Who should use this?

AI prompt engineers tweaking github ai prompt for production apps, especially in Node.js stacks. Teams A/B testing prompts for Copilot, Claude, or ChatGPT integrations who want prompt versioning meaning control without enterprise tools. Solo devs or small groups avoiding prompt github leak risks by versioning locally before pushing to prompt github repo.

Verdict

Try it for lightweight prompt versioning and A/B if you're in TypeScript—solid CLI gets you 80% there fast. With 11 stars and 1.0% credibility score, it's raw (thin docs, no tests), so fork and harden for real use; great prototype otherwise.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.