Abstract
Optimal control theory provides a systematic approach to control robotic systems. However, the applicability of optimal controllers to robots is limited by model-bias and excessive computation. Models are essentially inexact and inexact models lead to increased cost compared to the theoretically predicted cost, evaluated by presuming an exact model. Model-bias refers to the detrimental effect of inexact models in optimal control. There are two effective methods to address the aforementioned limitations. Model Predictive Control methods reduce the model-bias by re-optimizing the control inputs at each time step, but the re-optimization within a time step requires excessive online computation. Reinforcement learning methods reduce model-bias by learning a better model, but learning an adequate model requires a large amount of training data, which is cannot be afforded in many robot control applications. In this dissertation we present three new online optimal feedback controller methods to overcome the aforementioned limitations, affecting state-of-the-art applied optimal control methods. In our first work, we propose two efficiently computable locally optimal feedback controllers. The first controller is represented by a small-scaled constrained quadratic program solved at each time step; the coefficients of the quadratic program are evaluated along the optimal trajectory predicted by the model. The second controller is similarly represented by constrained quadratic programming but the coefficients of this quadratic program are re-evaluated using online measured state information. Compared to alternative optimal feedback and model predictive controllers, both of the aforementioned controllers can be efficiently computed in real-time by taking control constraints rigorously into account. In our second work, we propose a hardware-in-the-loop iterative optimal feedback controller, designed for nonlinear systems with constrained control inputs and inexact models. The proposed iterative optimal feedback controller combines an inexact model with measured state information such that the control input is always optimized along the measured trajectory of the controlled system; the model is never used for trajectory prediction. Compared to alternative model predictive methods that use model-based prediction, the proposed optimal feedback controller is demonstrated to provide improved performance and robustness against model uncertainty, thereby, addressing the long-standing limitation of theoretical optimal control methods that use inexact model-based future prediction. In our third work, we establish the theoretical foundation of a novel data-driven applied optimal control algorithm that is computationally efficient, and does not require model-based future prediction. This work presents the convergence and optimality properties of one such algorithm, not only when the system model is exact but also when the model is inexact. We formally prove the improved performance and robustness against model uncertainty of the proposed optimal control method for a scalar unconstrained linear-quadratic optimal control problem. Finally, we demonstrate that the new algorithm (i) is efficiently computable, (ii) can optimally take control constraints into account, and (iii) can effectively mitigate the detrimental effect of model-bias, compared to state-of-the-art optimal control methods. Finally the benefit of the proposed algorithm is demonstrated using a nonlinear optimal control problems where we assume unknown and changing dynamics. In summary, the online optimal feedback controllers presented in this dissertation can be efficiently computed and they lead to improved performance compared to alternative optimal control methods which use the same inexact model.