georgeguimaraes

georgeguimaraes / hallmark

Public

Hallucination detection for Elixir, powered by Vectara's HHEM model

bumblebee elixir hallucination-detection machine-learning nlp
10
0
100% credibility
Found Mar 07, 2026 at 10 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Elixir
AI Summary

Hallmark is an Elixir library that scores how well AI-generated text aligns with source material by evaluating consistency between premises and hypotheses.

How It Works

1
🔍 Discover Hallmark

You stumble upon Hallmark, a friendly helper that checks if AI-written answers truly match the facts you provide.

2
📱 Jump into the Demo

Click the easy 'Run in Livebook' button to try checking AI text pairs right away in a fun interactive playground.

3
🧠 Wake Up the Checker

The tool grabs its clever thinking brain once (a quick download), ready to spot made-up info in seconds.

4
📝 Test Your Texts

Share a key fact and what the AI said about it, getting back a score from 0 (dreamed up) to 1 (right on target).

5
📊 Check Lots at Once

Feed it a bunch of fact-AI pairs together and receive scores for all, speeding up your reviews.

6
✅ See Simple Labels

Ask for easy yes/no tags like 'consistent' or 'hallucinated' to quickly know if AI stayed truthful.

🎉 Trust Your AI More

With reliable checks, your AI creations now stick to the facts, making everything more dependable and honest.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 10 to 10 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is hallmark?

Hallmark brings github hallucination detection to Elixir apps, scoring how well LLM-generated hypotheses match source premises on a 0-1 scale—0 means hallucinated, 1 means consistent. Powered by Vectara's HHEM model via Bumblebee, it runs fully local after a one-time 440MB download, with no API calls needed. Load the model once, then use simple functions for single scores, batch processing, or binary labels like `:consistent` or `:hallucinated` with custom thresholds.

Why is it gaining traction?

It stands out as a lightweight hallucination detection library for LLMs in the Nx ecosystem, clocking in at 170ms per score with EXLA acceleration—far faster than pure Elixir fallback. No vendor lock-in, cross-platform GPU support (CUDA or Metal), and seamless Livebook integration make it a quick win over cloud-based alternatives or heavier Python ports. Devs dig the entailment focus: it flags ungrounded logic, not just facts, tying into hallucination detection benchmarks.

Who should use this?

Elixir backend devs building RAG pipelines or Phoenix apps with LLM outputs, needing fast checks if AI text sticks to source docs. Livebook tinkerers prototyping hallucination detection in llms, or teams validating chatbot responses without external services. Skip if you're not in Elixir—it's niche for those chasing local, memory-efficient fine-tuned models.

Verdict

With 10 stars and 1.0% credibility score, Hallmark is early-stage but battle-ready: solid docs, Livebook demo, and MIT license make it worth a spin for Elixir ML workflows. Test it in a side project before prod—promising foundation, just needs community miles.

(178 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.