WisdomShell / ADG

What is ADG?

ADG is a Python toolkit for selecting high-quality instruction data via answer divergence, scoring instructions by sampling multiple responses from a base model like Llama or Qwen and measuring their geometric spread in embedding space. It solves the fixed-budget data selection problem for LLM instruction tuning, delivering top, middle, and bottom subsets across semantic clusters for better coverage on reasoning, knowledge, and coding tasks. Users get a full pipeline: distributed answer generation, embedding/clustering, scoring, training scripts, and lm-evaluation-harness benchmarks.

Why is it gaining traction?

Unlike single-reference scorers, ADG combines dispersion magnitude and shape anisotropy for robust, geometry-aware selection, consistently boosting performance under 10K budgets as shown in the ACL'26 main conference paper. Developers dig the end-to-end workflow with torchrun for multi-GPU generation/training and bin-wise proportional picking to avoid dense-region collapse. It's practical for public pools like Alpaca-GPT4 or WizardLM, with clear quick-start paths.

Who should use this?

ML engineers fine-tuning open models on instruction datasets, especially those hitting data quality walls in reasoning-heavy apps. Researchers reproducing data selection experiments or iterating on custom pools for coding/knowledge tasks. Teams with GPU clusters needing distributed pipelines beyond basic random sampling.

Verdict

Worth forking for ACL'26 repros or LLM data curation experiments—solid docs and pipeline despite 16 stars and 1.0% credibility score signaling early maturity. Update paths, watch GPU memory, and test on your pool before production; lacks broad tests but delivers research-grade results fast. (198 words)