See, Think, Act: Training Agents by Reinforcing Reasoning

See, Think, Act: Training Agents by Reinforcing Reasoning

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning tasks, but the emerging Large Agent Models (LAMs) faces unique challenges as these models learn to interact with dynamic environments. This talk explores the fundamental framework for understanding and improving agent decision-making across long-horizon multi-round interactions. We begin by formalizing agent reasoning as a Markov Decision Process (MDP) and introduce the Embodied Agent Interface, a standardized framework for studying core agent capabilities including goal interpretation, subgoal decomposition, action sequencing, and transition modeling. Through this lens, we identify long-horizon decision making as a critical bottleneck that requires specialized training approaches. To address this challenge, we present RAGEN, a novel framework that is inspired by the recent success of DeepSeek-R1(Zero) using rule-based reward in reinforcement learning. RAGEN tackles two key challenges in real-world agent scenarios: environmental non-deterministic reward and long-horizon multi-turn interactions. To handle visual states, we introduce VAGEN to formulate the problem as a Partially Observable Markov Decision Process, enabling more robust learning in complex visual states.

Bio:Manling Li is an Assistant Professor at Northwestern University. She was a postdoc at Stanford University and obtained the PhD degree in Computer Science at University of Illinois Urbana-Champaign in 2023. She works on the intersection of language, vision, and robotics. Her work won the ACL'25 Inaugural Dissertation Award Honorable Mention, ACL’24 Outstanding Paper Award, ACL'20 Best Demo Paper Award, NAACL'21 Best Demo Paper Award, etc. She was a recipient of Microsoft Research PhD Fellowship in 2021, an EE CS Rising Star in 2022, etc. She served as Organizing Committee of ACL 25, NAACL 25, EMNLP 24, and delivered tutorials about multimodal knowledge at IJCAI'24, CVPR'23, NAACL'22, AAAI'21, ACL'21, etc. Additional information is available at https://limanling.github.io/.