Reinforcement learning based hybrid controller for robotic manipulator

Abstract

Reinforcement learning provides robotics with a framework and a set of tools to develop complex and challenging behaviors that are difficult to engineer. Although reinforcement learning showed a significant improvement of applying into the robotics field, we could find only very few industrial applications. The intrinsic challenge of data inefficiency represents a significant obstacle that restricts the application of reinforcement learning algorithms in addressing practical problems in robotics. Looking back at the development of reinforcement learning over the past, it could be identified that applying it alone is not sufficient for real industrial application in robotics. Therefore, the proposed solution is trying to apply reinforcement learning in more innovative way by introducing it to hybrid the existing controller by taking the inspiration from muscle memory concept in human. This thesis reports on the development of hybrid solution to robotic manipulator with the use of reinforcement learning alone with classical deterministic controller. Here, we utilize both PILCO, a practical and data-efficient model-based policy search method in reinforcement learning, which operates without the need for expert knowledge, alongside a PD controller as the deterministic controller. These components function in parallel to create a hybrid system. This proposed controller switches the control when observability is not sufficient for the deterministic controller to act, and the RL takes control over and executes the learned policy to robot to take its usual maneuvers. The assessment involved conducting an experiment using a basic two-wheeled line-following robot. During the experiment, the robot's camera view was disabled halfway through to activate a hybrid mechanism transitioning into RL mode. Upon executing this test scenario, the RL mode based on PILCO demonstrated promising outcomes compared to other machine learning methods. Research further emphasizes the importance of hybrid AI systems to cope with any uncertainty or disastrous situation in future mission critical applications of robotics. Discussion also suggesting future enhancements that can add to this concept as further improvements.