ZJU-OmniAI / GFT

What is GFT?

GFT is a Python framework for group fine-tuning large language models, shifting from pure imitation learning to reward fine-tuning via unbiased group advantages and dynamic coefficient rectification. It fixes supervised fine-tuning flaws like single-path dependency causing entropy collapse and unstable gradients leading to forgetting, delivering a stable base for downstream RL like GRPO or PPO. Developers get ready-to-run scripts for math reasoning on datasets like NuminaMath-CoT, achieving strong results with just 10k samples.

Why is it gaining traction?

Unlike standard SFT pipelines that hit ceilings on reasoning benchmarks, GFT boosts data efficiency – beating 100k-sample SFT with 10k while preserving policy entropy for better RL synergy. Its advantages shine in dynamic fine-tuning setups, where rectification stabilizes training without losing knowledge injection. Built on scalable tools like verl, it offers plug-and-play recipes for Python-based LLM post-training.

Who should use this?

ML engineers tuning LLMs for math or multi-step reasoning tasks, especially those frustrated with SFT's memorization pitfalls. AI researchers prototyping reward models or GRPO/PPO flows on Qwen models. Teams handling group-based imitation-to-reward transitions in production fine-tuning.

Verdict

Worth trying for efficient reward fine-tuning in Python, particularly if you're chasing unbiased advantages and dynamic rectification on reasoning data. With 19 stars and 1.0% credibility score, it's early-stage – solid paper (ACL 2026 Findings) and scripts, but verify maturity via small runs before scaling.

(198 words)