🔥 MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Management

💡 The paper introduces MemGUI-Agent, a mobile GUI agent designed to address the limitations of existing agents on long-horizon tasks. Current agents struggle with retaining intermediate facts across many steps and app transitions, leading to unreliable performance. This limitation is attributed to the ReAct-style prompting approach, which passively accumulates per-step records, causing prompt explosion and dilution of critical cross-app facts.

To address this issue, the authors propose MemGUI-Agent, which uses proactive context management through Context-as-Action, or ConAct. ConAct casts context management as first-class actions emitted by the same policy that selects UI actions. This approach maintains three structured context fields: folded action history, folded UI state, and recent step record, preserving critical UI facts while keeping context compact.

The authors also introduce MemGUI-3K, a dataset with 2,956 trajectories and full ConAct annotations for supervised training and offline analysis. Training an 8B model on MemGUI-3K results in MemGUI-8B-SFT, an 8B MemGUI-Agent that achieves the best open-data 8B performance on MemGUI-Bench and generalizes to the out-of-distribution MobileWorld benchmark.

The contributions of the paper are threefold. Firstly, it identifies the limitations of existing mobile GUI agents on long-horizon tasks and attributes them to the ReAct-style prompting approach. Secondly, it proposes MemGUI-Agent with proactive context management through ConAct, which addresses the limitations of existing agents. Finally, it introduces MemGUI-3K, a dataset for supervised training and offline analysis, and demonstrates the effectiveness of MemGUI-8B-SFT, an 8B MemGUI-Agent trained on this dataset. The code, data, and trained models will be released to facilitate further research and development.

📅 Published on Jun 18

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.19926
• PDF: https://arxiv.org/pdf/2606.19926
• Project Page: https://memgui-agent.github.io/

🤖 Models citing this paper:
• https://huggingface.co/lgy0404/MemGUI-8B-SFT

📊 Datasets citing this paper:
• https://huggingface.co/datasets/lgy0404/MemGUI-3K

━━━━━━━━━━━━━━━━━━━━━━━━
@Machine_learn