YilunKuang / rectified-lp-jepa

What is rectified-lp-jepa?

This official GitHub repository delivers Python code for Rectified LpJEPA, a self-supervised learning method that trains image encoders to produce sparse, non-negative representations aligned to rectified generalized Gaussian distributions. Developers get pretraining scripts for CIFAR-100 and ImageNet-100 subsets using backbones like ResNet, ViT, and ConvNeXt, plus baselines such as SimCLR and VICReg—all configurable via YAML files and runnable with PyTorch Lightning. It solves the challenge of balancing sparsity for efficient models with preserved task performance through explicit l0 control and maximum-entropy regularization.

Why is it gaining traction?

Unlike standard SSL methods, it offers tunable sparsity-performance trade-offs via Rectified Distribution Matching Regularization, yielding highly sparse features without losing invariance across views. The solo-learn foundation provides seamless multi-GPU training, wandb logging, and linear probing, making experiments faster than from-scratch setups. Random projections and rectified MLPs enable quick hyperparameter sweeps for custom distributions.

Who should use this?

Computer vision researchers tuning SSL for edge deployment, where sparse embeddings cut inference costs. Teams replicating arXiv:2602.01456 or extending JEPA architectures for downstream tasks like classification on limited hardware.

Verdict

Grab it if you're into sparse SSL—the official code matches the paper with solid configs and docs—but with 39 stars and 1.0% credibility score, it's early-stage; expect tweaks for production. Strong repro potential over toy baselines.

(198 words)