xie-lab-ml / Lightning-Unified-Video-Editor-via-In-Context-Sparse-Attention

What is Lightning-Unified-Video-Editor-via-In-Context-Sparse-Attention?

This Python repo delivers a fast, open-source video editor built on diffusion transformers that edits source videos via text prompts—like adding a crown to a character's head while preserving motion and structure. It tackles the O(S²) cost of full self-attention on long joint source+generated token sequences by using in-context sparse attention, slashing compute without retraining. Run `python inference.py --input video.mp4 --prompt "your edit" --output result.mp4` on a single RTX 4090 for results in 32 steps, with TileLang or Triton backends.

Why is it gaining traction?

It tops EditVerse benchmarks (e.g., 0.289 CLIP-T score) and runs 2.8x faster than FlashAttention-2 at 65K tokens, enabling practical video edits on consumer GPUs. Pluggable sparse kernels and 80-step fine-tuning make it drop-in for Wan 2.2 models, while matching ICML 2026 papers github quality with arXiv preprint and HF weights. Devs grab it for attention breakthroughs without dense compute walls.

Who should use this?

Video ML researchers prepping ICML 2026 submissions or rebuttals, prototyping apps like personalized ad edits or AR overlays. Devs at startups building text-driven video tools, especially those eyeing ICML 2026 workshops on efficient transformers. Skip if you're not on H100/RTX 4090 or need multi-GPU scale yet.

Verdict

Solid pick for fast video editing inference if you're into ICML 2026 github experiments—benchmarks and speed impress—but 19 stars and 1.0% credibility score signal early-stage; read the ICML 2026 paper first, test demos thoroughly before production.

(198 words)