Seminar

Abstract

We propose a novel adaptive critic learning algorithm for a continuous-time nonlinear system subject to excitation and weight constraints. The algorithm is able to learn the optimal control in real-time under only finite excitation without requiring the a priori knowledge of the system model, i.e., the Hamilton-Jacobi-Bellman (HJB) equation is approximately solved online by the adaptive critic learning of a nonlinear Q-function.

The main contribution of this paper is twofold: First, we present an optimisation-based approach to the derivation of a weight-error-driven adaptive law that guarantees exponential convergence of the critic weight. Such formulation enables a new P-projection operator to enhance the convergence property, i.e., the weight estimate always stays in a bounded convex set that contains the true weight. Second, we adopt a new measure to build the information matrix that stores its richness over incoming data such that the standard persistent excitation (PE) condition is relaxed to a finite excitation (FE) condition. In this way, the convergence of the critic weight is guaranteed without persistently injecting exploration noise. We show that the method is model-free and can achieve semi-global stability. A numerical example demonstrates the effectiveness of the theoretical result.

Biography

Dr. Anthony Siming Chen received B.Eng. in Transportation Engineering from Central South University, China, in 2015. After moving to the United Kingdom, he studied at the University of Bristol with Professor Guido Herrmann as his advisor, and he received his M.Sc. and Ph.D. in (Advanced) Mechanical Engineering in 2017 and 2021, respectively. He was also a Visiting Researcher at the Institute of Advanced Propulsion Systems (IAAPS) at the University of Bath from 2015 – 2022.

From 2022, he has been working as a Postdoctoral Research Associate with the Department of Electrical and Electronic Engineering at the University of Manchester for RAIN+ and LongOps projects. His research interests include adaptive/optimal control, reinforcement learning, telerobotics, and automotive systems.

Abstract

Network-connected systems have received significant research attention over the past two decades, especially, in the domains of optimisation learning and control. Distributed algorithms have been developed aiming at facilitating decision-making at the local level while accomplishing certain global tasks through network communications.

The interaction between learning and control plays an important role in achieving intelligent operation for robotic systems in challenging environment. This presentation will focus on the recent advancement in the concept of reciprocal learning and control and its application in autonomous search problems.

Biography

Zhongguo Li received both B.Eng. and Ph.D. degrees in electrical and electronic engineering from the University of Manchester, Manchester, UK, in 2017 and 2021, respectively. He is currently a Lecturer in Robotics, Control, Communication and AI with the Department of Electrical and Electronic Engineering at the University of Manchester.

Before joining Manchester, he was a Lecturer in Robotics and AI at University College London and a Research Associate at Loughborough University. His research interests include optimisation and decision-making for advanced control, distributed algorithm development for game and learning in network connected multi-agent systems, and their applications in robotics and autonomous vehicles.

Abstract

Backlash and saturation are two common nonlinearities in physical systems and may result in poor control performance. In previous studies, a novel control strategy based on Saturation Equivalence(SE) and Model Predictive Control (MPC) has been proposed for plants subject to both saturation and backlash at their input. This paper analyzes the performance of the control strategy under parameter uncertainty and presents recommendations for its implementation.

Simulation experiments illustrate the theoretical analysis and quantify the performance of the control strategy. The results show that it is better to underestimate than to overestimate both the backlash gap and the saturation level.

Biography

Tengyang Gong received his MSc degree in Advanced Control and Systems Engineering from The University of Manchester in 2022, and his MSc dissertation was supervised by Prof. William P. Heath. He is currently a first-year PhD student in the control systems group at the University of Manchester, working on distributed optimization under the supervision of Prof. Zhengtao Ding.