EndoTheDev

EndoTheDev / OMeter

Public

Benchmark and compare Ollama models across local and cloud endpoints with rich, sortable tables.

benchmark cli ollama performance python
11
1
100% credibility
Found Apr 28, 2026 at 11 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

OMeter is a command-line tool that lists, benchmarks performance metrics like time-to-first-token and tokens-per-second, and compares Ollama AI models from local and cloud sources in interactive sortable tables with export options.

How It Works

1
📖 Discover OMeter

You hear about a simple tool that helps compare different AI models to see which ones are fastest and best for your needs.

2
🛠️ Get it ready

You add the tool to your computer in just a couple of clicks, and it's good to go anywhere on your machine.

3
🔗 Connect your AI setups

You jot down quick notes about where your local and online AI models live, so the tool knows where to look.

4
Choose your models
🏠
Local only

Focus on AI models running right on your own machine.

☁️
Cloud only

Look at AI models available over the internet.

🔄
Both

Compare models from your machine and the cloud side by side.

5
⚡ Test and see results

The tool runs quick tests on the models and shows colorful tables with speeds, sizes, and details updating live.

6
🔍 Sort and pick favorites

Easily rearrange the list by speed, size, or other details to spot the top performers.

7
💾 Save your findings

Copy the results to a simple file to keep or share with friends.

🎉 Find your best AI

You now know exactly which model is quickest and perfect for what you want to do next.

Sign up to see the full architecture

6 more

Sign Up Free

Star Growth

See how this repo grew from 11 to 11 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is OMeter?

OMeter is a Python CLI tool for listing, benchmarking, and comparing Ollama models across local endpoints (like your CPU or GPU on MacBooks, PCs, or tablets) and cloud services. Run `ometer --local --ttft --tps` to get sortable rich tables showing time-to-first-token, tokens-per-second, model size, context length, and more, averaged over multiple prompts. It solves the hassle of manually testing model performance to pick the best for your hardware or budget.

Why is it gaining traction?

Unlike basic Ollama wrappers, it delivers live-updating tables with color-coded percentiles (green for top performers), parallel benchmarking up to 10 models, family-name filtering (e.g., `--model llama3`), and JSON/CSV exports for further analysis. The interactive menu and verbose per-run breakdowns make spotting outliers easy, while embedding model support via `/api/embed` handles specialized cases. Developers grab it for quick AI model benchmarks without scripting from scratch.

Who should use this?

Ollama users tweaking local setups—think ML engineers benchmark comparing AI models on Apple silicon vs. NVIDIA GPUs, or devs optimizing for GitHub Actions CI runs. Ideal for hardware evaluators pitting MacBook CPUs against PC GPUs, or edge deployers testing tablets and low-power devices. Skip if you're not running Ollama.

Verdict

Worth installing via `uv tool install` for any serious Ollama workflow; docs are thorough, tests cover the basics, and MIT license keeps it low-risk despite 11 stars and 1.0% credibility score signaling early maturity. Test it on your rig before depending on it.

(178 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.