MatthewK78 / Rose

What is Rose?

Rose is a Python PyTorch optimizer that performs stateless gradient updates via range-of-slice equilibration, normalizing each output neuron's gradient by its max-min spread across input dimensions. It ditches the memory-heavy buffers and history tracking of Adam or RMSprop, letting you train with just parameters plus gradients. Pip-install from GitHub and use it like `optimizer = Rose(params, lr=1e-3)` for immediate drop-in optimization.

Why is it gaining traction?

Zero optimizer state slashes memory use, ideal when stateful alternatives double your footprint during large-model training. Built-in gradient centralization, coefficient-of-variation trust gating for noisy ranges, decoupled weight decay, BF16 stochastic rounding, and FP64 compute precision handle stability and low-precision pitfalls out of the box. Developers dig the "just current gradient" simplicity amid gradient equilibration hype, echoing ideas from alban rose github experiments.

Who should use this?

PyTorch ML engineers fine-tuning LLMs or diffusion models on tight VRAM budgets. Researchers benchmarking optimizers beyond gilded rose github baselines or rose pine neovim distractions. Anyone chasing faster convergence without Adam's tuning rituals in transformer pretraining.

Verdict

Worth a benchmark for memory-constrained gradient-heavy workflows, especially with its clean docs and features like schedule-coupled decay. But 14 stars and 1.0% credibility scream early-stage—prototype only, no prod bets yet.

(178 words)