YutoTerashima

YutoTerashima / agent-safety-eval-lab

Public

Agent trace and tool-use safety evaluation lab.

ai-agents evals llm-safety red-teaming tool-use
46
2
100% credibility
Found May 03, 2026 at 44 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

A lab for reproducibly evaluating AI agent safety through mock and real traces, tool policies, and classification benchmarks on public datasets.

How It Works

1
🔍 Discover the Safety Lab

You find this handy lab online that helps check how safe AI helpers are when they use tools and chat with people.

2
🛠️ Set Up Your Test Space

You make a cozy spot on your computer by following simple steps to prepare everything for testing.

3
🚀 Run a Quick Safety Check

You launch a demo test on pretend AI behaviors and instantly see which ones pass or raise red flags.

4
📊 Review Your First Report

Colorful charts and summaries pop up, showing safe actions, blocked risks, and what needs a closer look.

5
🔬 Test Real-World Scenarios

You try tests with actual conversation examples to spot hidden dangers in AI responses.

6
📈 Dive Into the Details

Explore lists of failures, risk scores, and patterns to understand exactly what's going wrong.

✅ Build Safer AI

You gain clear insights to make your AI helpers smarter and safer, ready for real use.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 44 to 46 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is agent-safety-eval-lab?

This Python repo is a lab for evaluating LLM agent safety through full traces: messages, tool calls, policy checks, and risk outcomes. It simulates agent runs in mock mode by default, then grades trajectories for tool violations, unsafe content, and budgets, producing metrics like pass rates and risk scores. Developers get CLI commands like `python -m agent_safety_eval_lab.cli run-demo` or `replay` for JSON traces, plus GPU-backed experiments on datasets like BeaverTails for agent trace analysis.

Why is it gaining traction?

It stands out with a mock-to-real pipeline that normalizes traces from OpenAI Agents, LangGraph, or local models into one grader—no SDK rewrites needed. The V2 research suite spits out publishable reports, figures, and failure analysis on 50k safety examples, beating basic keyword rules with TF-IDF classifiers hitting 0.77 macro-F1. For agent GitHub repos building Claude or Copilot-like tools, it's a quick way to benchmark trace visualization and policy simulation without starting from scratch.

Who should use this?

Agent builders testing tool-use safety in production pipelines, like teams behind GitHub Copilot CLI or VSCode extensions handling risky calls. AI safety researchers running evals on agent trace GitHub datasets, or devs at GitHub Agent HQ needing reproducible GPU benchmarks for traceability. Ideal for red-teaming agent trace cursor or Lego-style workflows before deployment.

Verdict

Grab it if you're deep into agent eval—solid docs, pytest coverage, and artifacts make reproduction easy despite 32 stars and 1.0% credibility score. Fork for custom policies; it's raw but battle-tested on real safety data.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.