ECE Seminar: Derya Aksaray

ECE Seminar: Reinforcement Learning for Dynamical Systems with Temporal Logic Specifications

Derya Aksaray

Location: 442 Dana or Zoom Link

Abstract: In many applications, dynamical systems such as drones, mobile robots, or autonomous cars need to achieve complex specifications on their trajectories which may include spatial (e.g., regions of interest), temporal (e.g., time bounds), and logical (e.g., priority, preconditions, concurrency among tasks) requirements. As these specifications become more complex, encoding them via algebraic equations become intractable. Alternatively, such specifications can be compactly expressed and used in control synthesis by utilizing the framework of temporal logics. In this talk, I will address the problem of learning optimal control policies for satisfying temporal logic (TL) specifications in the face of uncertainty. Standard reinforcement learning (RL) algorithms, which aim to maximize the expected sum of discounted rewards, are not directly applicable when the objective is to satisfy a TL specification. To overcome this limitation, I will formulate an approximate problem that can be solved via reinforcement learning and present the suboptimality bound of the proposed solution. Then, I will consider the case where a TL specification is given as the constraint rather than the objective and present a novel approach for satisfying the TL constraint with a desired probability throughout the learning process. I will motivate this part by multi-use of autonomous systems, e.g., a drone executing a pick-up and delivery mission as its primary task (constraint) while learning to fly over regions of interest (aerial monitoring) as its secondary task (objective). Finally, I will conclude my talk by discussing some future directions toward the resilience and safety of autonomous systems with complex specifications.

Bio: Derya Aksaray is currently an Assistant Professor in the Department of Aerospace Engineering and Mechanics at the University of Minnesota (UMN). Before joining UMN, she held post-doctoral researcher positions at the Massachusetts Institute of Technology from 2016-2017 and at Boston University from 2014-2016. She received her Ph.D. degree in Aerospace Engineering from the Georgia Institute of Technology in 2014. Her research interests lie primarily in the areas of control theory, formal methods, and machine learning with applications to autonomous systems and aerial robotics.