DestinyLinker / MingLi-Bench

What is MingLi-Bench?

MingLi-Bench is a Python CLI benchmark evaluating large language models on Chinese traditional fortune telling—Bazi and Ziwei Doushu—with 160 multiple-choice questions from global competitions (2022-2025). It tests LLMs on life predictions like career, marriage, and wealth, using exact-match scoring and options like Chain-of-Thought prompting or pre-computed astrological charts to isolate reasoning from chart generation. Run `python -m mingli_bench.cli --model gpt-4o --cot --astro` for instant results with JSON summaries and raw responses.

Why is it gaining traction?

This stands out in the bazi benchmark space by supporting OpenRouter for one-key access to dozens of models, plus native APIs for DeepSeek, Doubao, and others—filter by year or category like `--categories 事业 婚姻`. Developers love the concurrency (`--max-workers 8`), option shuffling to kill position bias, and artifacts like per-question files for debugging LLM failures. It's a quick way to probe cultural reasoning gaps without building your own eval suite.

Who should use this?

LLM researchers benchmarking models on non-Western domains, Chinese AI teams evaluating local providers like Doubao on traditional knowledge, or anyone auditing frontier models for fortune-telling accuracy. Ideal for ablation studies on CoT or astrology injection in prompts.

Verdict

Grab it if you're into niche LLM evals—solid CLI and docs make it dead simple, despite 30 stars and 1.0% credibility signaling early days. MIT-licensed and production-ready for quick tests, but watch for dataset expansions.

(198 words)