dongyh20

dongyh20 / Demo-ICL

Public

Demo-ICL: In-Context Learning for Procedural Video Knowledge Acquisition

34
0
100% credibility
Found Feb 10, 2026 at 27 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

Demo-ICL provides a benchmark and model to test AI's ability to learn procedural tasks from text or video demonstrations using in-context learning.

How It Works

1
🔍 Discover Demo-ICL

You stumble upon this project while reading about smart AI that learns from video tutorials, like figuring out the next step in a recipe.

2
📖 Explore the idea

It shows how AI watches short clips or reads instructions to predict what happens next in everyday tasks, exciting for testing real-world smarts.

3
🎥 Grab example videos

Download simple video snippets of common activities, like cooking or fixing things, to see AI learn on the fly.

4
🤖 Pick your AI helper

Choose a vision-language AI you've heard of, ready to challenge it with these learning tests.

5
Choose learning style
📝
Text steps

AI reads instructions to guess the next move.

📹
Video demo

AI watches an example clip to copy the actions.

6
▶️ Run the challenge

Watch the AI tackle new videos, predicting steps based on what it learned.

✅ See the scores

Get clear results showing how well your AI picks up practical skills from demos, ready to share or improve.

Sign up to see the full architecture

5 more

Sign Up Free

Star Growth

See how this repo grew from 27 to 34 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is Demo-ICL?

Demo-ICL is a Python-based benchmark and evaluation suite for testing multimodal large language models on in-context learning from procedural video demos, like predicting next steps in HowTo100M instructional clips. It provides Demo-ICL-Bench with 1,200 samples across text-demo, video-demo, and demo selection tasks, letting you assess dynamic knowledge acquisition without static training. Users get a CLI-powered framework via lmms-eval to run evals on 30+ video models, plus a 7B SOTA model fine-tuned on Ola-Video for superior demo utilization.

Why is it gaining traction?

Unlike static video QA benchmarks, Demo-ICL focuses on transferable procedural knowledge from few-shot demos, revealing negative transfer issues in baselines while its model gains +14% from text demos. lmms-eval stands out with 90+ tasks, video/audio support, and acceleration via vLLM or SGLang, making it faster than ad-hoc scripts. Developers hook into its arXiv-backed pipeline for quick ICLR demo track-style experiments on video ICL.

Who should use this?

Video AI researchers benchmarking MLLMs for robotics or instructional agents, where procedural learning from demos matters. Multimodal eval teams tired of fragmented tools for models like LLaVA-NeXT-Video or Qwen2-VL. Python devs prototyping video knowledge acquisition in apps like tutorial analyzers.

Verdict

Worth forking for video ICL evals if you're in multimodal research—its benchmark fills a gap—but at 18 stars and 1.0% credibility, treat as early-stage; pair with mature lmms-eval docs and test your setups first. (198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.