lechmazur

lechmazur / sycophancy

Public

LLM benchmark and leaderboard for narrator-bias sycophancy, opposite-narrator contradictions, and judgment consistency.

benchmark consistency contradiction evaluations leaderboard
14
0
100% credibility
Found Mar 16, 2026 at 14 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
AI Summary

This repository hosts a benchmark evaluating how consistently large language models judge the same disputes when narrated from opposing first-person perspectives.

How It Works

1
🔍 Discover the AI Fairness Test

You stumble upon this clever test that checks if AI chatbots play favorites based on who's telling the story.

2
📊 Browse the Leaderboards

You scan colorful charts ranking popular AI models from fairest to most swayed by emotions and viewpoints.

3
💡 Spot the Standouts

You notice top performers like Gemini staying steady, while others flip-flop to agree with whoever speaks.

4
📖 Learn How It Works

You read simple explanations of disputes told from opposite sides to reveal if AIs bend to the narrator.

5
🔍 Explore Real Examples

You dive into everyday arguments like roommate messes, seeing exactly how each AI judges from both angles.

6
📈 Uncover Hidden Patterns

You discover trends, like some AIs abstaining wisely or getting emotional framing wrong.

✅ Master AI Biases

Now you understand which AIs keep fair judgments no matter the story, empowering smarter choices.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 14 to 14 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is sycophancy?

This GitHub llm-resources repo runs an llm benchmark 2026 focused on narrator-bias sycophancy in LLMs: it feeds the same dispute from opposite first-person views—neutral, stripped, and affective—and measures if models flip judgments to agree with whoever speaks, tracking sycophancy ai, contrarian flips, and consistency. Users get detailed llm benchmark leaderboard tables and charts ranking 16 top models like Gemini 3.1 Pro Preview (0.5% sycophancy) and Grok betas, plus breakdowns on decisive coverage and insufficient responses for llm benchmark comparison. It's a pure docs project in unknown language, delivering instant llm benchmark arena insights without setup.

Why is it gaining traction?

It stands out in llm benchmarking with a strict headline metric—sycophancy only when models side with both opposing emotional narrators—plus a consistency view counting total contradictions, revealing hidden flaws like high contrarian rates in "low sycophancy" leaders. Developers hook on the 199-case dataset across workplace to privacy topics, visualizations like affective uplift and net narrator pull, and ties to related llm github repositories for sycophancy chatgpt eval. No cli or api yet, but the leaderboards enable quick llm github integration for custom tests.

Who should use this?

LLM researchers benchmarking judgment consistency for ethical AI or legal apps, AI teams comparing models like Claude vs GPT-5.4 in llm benchmark tests, and fine-tuners spotting sycophancy deutsch in narrative tasks. Ideal for devs building llm github copilot features needing stable rulings under biased prompts, or running llm github actions for llm benchmark questions.

Verdict

Solid conceptual benchmark with excellent docs and fresh 2026 data, but at 14 stars and 1.0% credibility score, it's early-stage—track for updates rather than production use. Grab it from the llm github repository for quick model scouting.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.