xie-lab-ml

xie-lab-ml / Mano-Restriking-Manifold-Optimization-for-LLM-Training

Public

The official code of "Mano: Restriking Manifold Optimization for LLM Training".

arxiv.orgabs2601.23000
14
2
100% credibility
Found Feb 04, 2026 at 12 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

Mano is an optimizer that improves the efficiency and performance of training large language models by applying manifold optimization techniques.

How It Works

1
🔍 Discover Mano

You hear about Mano, a clever tool that makes training big AI language models faster and smarter with less effort.

2
📖 Explore the page

You read the simple guide and check out the charts showing how Mano beats usual methods in speed and results.

3
📊 See the proof

The graphs light up your eyes, proving Mano delivers quicker learning and steadier progress during training.

4
🔧 Add to your setup

You copy the easy example and slip Mano into your AI model's training routine.

5
▶️ Start training

You launch the training, and Mano takes over to guide your model smoothly.

6
📈 Watch it shine

You see the training zoom ahead with better stability, less memory drain, and top performance.

🎉 Training triumph

Your AI model learns faster and stronger, ready to tackle tough tasks brilliantly.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 12 to 14 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is Mano-Restriking-Manifold-Optimization-for-LLM-Training?

This official GitHub repository delivers the official code for Mano, a Python-based PyTorch optimizer designed to train large language models more effectively. It tackles the limitations of standard optimizers like AdamW by projecting updates onto a manifold, delivering faster convergence and better performance on benchmarks like LLaMA models trained on Pile data. Developers get a drop-in replacement that splits parameters—using Mano for core weights and AdamW for embeddings/heads—via simple init calls with learning rate, momentum, and weight decay.

Why is it gaining traction?

Mano stands out by closing the gap between manifold methods and heavyweights like AdamW or Muon, showing superior loss curves, gradient stability, and signal-to-noise ratios in demos, all with lower memory and compute. The hook is its efficiency for LLM fine-tuning: plug it into your training loop for immediate gains without rewriting models. As the official GitHub release for the arXiv paper, it draws devs eyeing cutting-edge optimizers beyond official GitHub Copilot or Actions defaults.

Who should use this?

LLM trainers scaling models on limited GPUs, like researchers fine-tuning LLaMA variants on custom datasets. Teams optimizing distributed training pipelines where every update counts, especially those already using PyTorch and frustrated with AdamW's plateauing. Not for casual scripters—ideal for ML engineers chasing perplexity drops in production setups akin to official code zones in Roblox or Blox Fruits projects.

Verdict

Promising for experimental LLM workflows, with solid example usage and paper-backed results, but at 13 stars and 1.0% credibility score, it's early-stage—docs are basic, no tests visible. Try it on small runs if you're prototyping optimizers; skip for mission-critical deploys until more adoption.

(178 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.