Title

Planning and learning for partially observed systems

Résumé

Reinforcement learning (RL) provides a conceptual framework for designing agents that learn to act optimally in unknown environments. Reinforcement learning has been successfully used in applications ranging from robotics and industrial automation to finance, healthcare and natural language processing. The success of RL rests on a solid foundation combining the theory of exact and approximate Markov decision processes (MDPs) with iterative algorithms that guarantee the learning of an exact or approximate action-value function and/or an approximately optimal policy. However, most research into the theory of Markov decision processes focuses on systems with complete state observations.

In various applications, including robotics, finance and healthcare, the agent obtains only a partial observation of the state of the environment. In this talk, I will describe a new framework for approximate planning and learning for partial observation systems, based on the notion of approximate information state. The talk will highlight the sound theoretical foundations of this framework, illustrate how many existing approximation results can be considered as a special case of approximate information state, and provide strong empirical evidence to show that this approach works well in practice.

Joint work with Jayakumar Subramanian, Amit Sinha, Raihan Seraj, and Erfan Seyedsalehi.

Biography

Aditya Mahajan is Professor of Electrical and Computer Engineering at McGill University, Montreal, Canada, and DATAIA Visiting Professor at the Signal and Systems Laboratory, Centrale-Supelec, Université Paris-Saclay. He is a member of McGill's Centre for Intelligent Machines (CIM), the AI Institute of Quebec (Mila), the CNRS International Laboratory for Learning Systems (ILLS) and the Groupe d'études et de recherche en analyse des décisions (GERAD). He holds a bachelor's degree in electrical engineering from the Indian Institute of Technology, Kanpur, India, and master's and doctoral degrees in electrical engineering and computer science from the University of Michigan, Ann Arbor, USA.

He is currently Associate Editor of IEEE Transactions on Automatic Control, IEEE Control Systems Letters and Springer Mathematics of Control, Signal, and Systems. He served as Associate Editor of the IEEE Control Systems Society Conference Editorial Board from 2014 to 2017.

He received the George Axelby Outstanding Paper Award in 2015, the NSERC Discovery Accelerator Award in 2016, the CDC Best Student Paper Award in 2014 (as supervisor) and the NecSys Best Student Paper Award in 2016 (as supervisor). His research focuses on decentralized stochastic control, team theory, reinforcement learning, multi-armed bandits and information theory.