Low-Level control of small scale helicopter using Soft Actor-Critic method

  • Author / Creator
    Kamyab, Majid
  • Unmanned Aerial Vehicles (UAVs), or drones, have been employed in a variety of applications, ranging from surveillance to emergency operations. These systems comprise an ”inner loop” that provides stability and control and an ”outer loop” in charge of mission-level tasks, such as way-point navigation. Despite their inherent instability, different techniques for controlling these robots have been devised under stable environmental conditions. However, these algorithms must know a robot’s dynamics to be effective; furthermore, more complex control is necessary for UAVs to perform in unstable environmental conditions. In this research, a simulated drone has been successfully controlled using model-free reinforcement learning with no prior knowledge of the robot’s model. Soft Actor-Critic (SAC) method is trained to perform low-level control of a small-scaled helicopter in a set-point control system. First, a simulation environment is created in which all tests were carried out and then it is shown that SAC can not only develop a strong policy, but it can also deal with unknown circumstances. The result obtained by the SAC agent is also compared to a sliding mode controller to compare the capability of this method to a traditional nonlinear control method. The SMC method proved to be superior by a steady state error of 0, compared to a steady state error of 0.05% for the SAC agent. However, the SAC agent is a model free technique which does not have access to the model of the helicopter, on the other hand the SMC is a model based technique whuch needs the system identification of the helicopter system.

  • Subjects / Keywords
  • Graduation date
    Fall 2021
  • Type of Item
  • Degree
    Master of Science
  • DOI
  • License
    This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.