1Password

1Password / SCAM

Public

SCAM - Security Comprehension Awareness Measure | Open-source benchmark that tests AI agents' security awareness during realistic, multi-turn workplace tasks.

1Password.github.ioSCAM
91
4
100% credibility
Found Feb 13, 2026 at 29 stars 3x -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

SCAM is an open-source benchmark created by 1Password to evaluate how well AI agents detect phishing, protect credentials, and avoid security pitfalls in everyday workplace tasks.

How It Works

1
🏆 Discover the safety test

You visit the leaderboard website to see how popular AI assistants handle everyday security threats like phishing emails and credential leaks.

2
📥 Get the testing tool

Download the free tool that lets you test any AI assistant on realistic workplace scenarios without needing technical skills.

3
🤖 Pick your AI helpers

Choose from your favorite AI models like Claude or GPT, and the tool guides you through connecting them securely.

4
🚀 Run the safety benchmark

Hit start and watch as the tool simulates inbox checks, web browsing, and credential use to reveal how well each AI spots dangers.

5
📊 Review scores and replays

See clear safety percentages, watch video replays of what went wrong or right, and compare models side-by-side.

6
🛡️ Boost security with tips

Apply the simple security prompt provided to teach your AI better habits, then retest to measure improvement.

✅ Safer AI assistant ready

Your AI now handles threats like a pro, protecting your emails, logins, and data in real work situations.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 29 to 91 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is SCAM?

SCAM is a Python CLI benchmark from 1Password that drops AI agents into realistic workplace simulations—checking phishing-filled inboxes, searching credential vaults, or filling web forms—to test if they spot threats like credential exposure or e-commerce scams before acting. Unlike single-shot "is this phishing?" tests, it runs multi-turn tasks with sandboxed tools for email, browsers, and vaults, scoring proactive safety via checkpoints. Users get leaderboards, video replays, and a plug-and-play security prompt skill that boosts model scores from 50% to 90% on scam detection.

Why is it gaining traction?

It stands out with interactive CLI runs (`scam evaluate -i`), HTML dashboards, and MP4 exports of failures, making it dead simple to benchmark models like GPT-4o or Claude on real threats like github scam links or discord scam github bots. The security skill installs via `npx add-skill` for 35+ agents, delivering instant wins on scam sniffer github tasks without custom coding. Public leaderboards expose weak spots in popular LLMs, hooking AI builders chasing enterprise-grade safety.

Who should use this?

AI engineers integrating LLMs with tools for email automation or credential managers, red-teamers probing prompt injection in multi-stage attacks, and security leads vetting models against inbox phishing or social engineering scams. Perfect for devs building scam detector github bots or evaluating agents handling scam security alert emails from Microsoft or Instagram.

Verdict

Grab it if you're building secure AI agents—solid docs, MIT license, and replay tools make early testing worthwhile despite 18 stars and 1.0% credibility score signaling newness. Run a baseline vs. skill eval today; it'll expose risks fast.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.