JetAstra

JetAstra / MacAgentBench

Public

MacAgentBench: Benchmark agents where they actually work — on macOS.

18
0
100% credibility
Found Mar 19, 2026 at 18 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

MacAgentBench provides a Dockerized macOS environment to benchmark AI agents on 110 realistic desktop tasks across apps like Notes, Reminders, and Keynote.

How It Works

1
🔍 Discover MacAgentBench

You find this project while looking for ways to test AI helpers on everyday Mac apps like Notes or Keynote.

2
📥 Grab the ready Mac setup

Download a complete Mac environment with AI tools already installed—no setup hassle.

3
🚀 Start your virtual Mac

Run a simple command to launch a full Mac desktop in a window on your computer.

4
🖥️ Connect and peek inside

Use a screen viewer to see your Mac desktop come alive, with AI assistant ready to go.

5
Choose your adventure
🧪
Run benchmark tests

Pick AI models and let them tackle 110 real Mac chores like reminders or slides.

🖱️
Explore interactively

Chat with the AI to make it edit notes or check weather right on the Mac screen.

6
📹 Watch magic happen

See the AI click, type, and complete tasks just like a human would on your Mac apps.

🏆 Get scores and celebrate

Review pass rates, video recordings, and compare your AI on the live leaderboard.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 18 to 18 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is MacAgentBench?

MacAgentBench is a Python-based benchmark for evaluating AI agents on real macOS desktop tasks, like managing Notes, Reminders, or Keynote slides. It runs reproducible scenarios in a Dockerized macOS environment accessible from Linux or Windows, using rule-based evaluators to score success without flaky LLM judges. Developers get a quick way to test if agents actually work where users live—on macOS.

Why is it gaining traction?

Unlike Linux-focused benchmarks, MacAgentBench hits 110 tasks across 18 native macOS apps, bridging the gap between eval sandboxes and daily workflows. The out-of-the-box Docker image and live leaderboard make it dead simple to spin up evals for models like OpenClaw or Claude, with bash scripts for batch runs. Rule-based scoring ensures reliable, interference-free results per task.

Who should use this?

AI researchers tuning vision-language agents for desktop automation, especially OpenClaw users validating macOS skills. Agent devs building tools for productivity apps like Pages or Safari, or teams comparing models on GUI-heavy tasks like email triage in Himalaya or GIF searches in GifGrep.

Verdict

Grab it if you're serious about macOS agent benchmarks—early promise with solid Docker setup and broad task coverage, despite just 18 stars and 1.0% credibility score signaling room for more tests and contributors. Maturity is low, but docs and quick-start guide it toward production use.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.