lgy0404

lgy0404 / MemGUI-Bench

Public

Official code repo for the paper "MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments"

lgy0404.github.ioMemGUI-Bench
26
4
100% credibility
Found Feb 09, 2026 at 19 stars -- GitGems finds repos before they trend. Get early access to the next one.
Sign Up Free
AI Analysis
Python
AI Summary

MemGUI-Bench is a benchmark for evaluating AI agents' long-term memory in dynamic Android app interactions across 128 tasks in 26 apps.

How It Works

1
🔍 Discover MemGUI-Bench

You find this helpful tool through a research paper or website that tests how smart phone helpers remember what they've seen on apps.

2
📱 Set up your phone playground

You get a pretend phone screen ready using a simple box or your own computer setup so tests can run smoothly.

3
🤖 Pick and connect your helper

Choose an AI buddy and link it up so it can see and act on the phone screens during tests.

4
🚀 Start the memory challenges

Hit go to run tests where your helper navigates real apps like contacts or camera, remembering steps across screens.

5
📊 Watch results roll in

See live updates on success rates, memory strength, and recovery from mistakes as tests finish one by one.

🏆 Review scores and share

Check detailed reports on how well your helper remembers and competes, then share on the public rankings to compare.

Sign up to see the full architecture

4 more

Sign Up Free

Star Growth

See how this repo grew from 19 to 26 stars Sign Up Free
Repurpose This Repo

Repurpose is a Pro feature

Generate ready-to-use prompts for X threads, LinkedIn posts, blog posts, YouTube scripts, and more -- with full repo context baked in.

Unlock Repurpose
AI-Generated Review

What is MemGUI-Bench?

MemGUI-Bench is the official GitHub repository for benchmarking memory in mobile GUI agents on dynamic Android environments, using Python with Android emulators. It runs 128 tasks across 26 apps and 68 scenarios—89.8% memory-intensive, averaging 36 steps and spanning 1-4 apps—delivering metrics like Pass@K, IRR, FRR, and MTPR. Users execute agents via simple CLI commands like `python run.py`, get real-time progress, and submit results to the leaderboard or HuggingFace datasets.

Why is it gaining traction?

Docker image handles emulator setup out-of-the-box, skipping Android SDK pains, while config.yaml tweaks parallelism and API keys for quick agent swaps. Comprehensive evals cover info retrieval, failure recovery, and cross-app navigation, with leaderboard-ready JSON exports. Official GitHub actions and release mirrors make sharing reproducible runs effortless.

Who should use this?

Researchers evaluating VLMs like Qwen-VL or UI-TARS on mobile tasks needing long-term memory. Agent devs testing robustness in apps like Contacts or SMS with dynamic UI changes. Teams comparing baselines like SeeAct before deploying to production Android automation.

Verdict

Grab it if you're building memory-aware GUI agents—docs and Docker shine despite 21 stars and 1.0% credibility signaling early days. Run the 40-task subset first to validate; submit your official GitHub release results to climb the leaderboard.

(198 words)

Sign up to read the full AI review Sign Up Free

Similar repos coming soon.