Pose estimation of a robot arm from a single camera

Abstract

This paper describes a vision based deep learning approach to estimate the pose of a robot arm from a single camera input, without any depth information. Conventionally, pose of robot arm is determined using encoders which sense the joint angles, and then the pose of each link (including the end effector) relative to the robot base is obtained from the direct kinematics of the manipulator. But there may be inaccuracies in the determined pose when the encoders or the manipulators are malfunctioning. This paper presents an approach based on computer vision, where a single RGB camera is fixed at a distance from the robot arm. Based on the kinematics of the manipulator and the calibrated camera, the input 2-dimensional image is reconstructed in 3-dimensional form and the pose of the manipulator is determined by means of a deep network model trained on synthetic data. Furthermore, a graphical user interface (GUI) is developed, which simplifies the output interpretation for users who operate the implemented system. Finally, the effectiveness of the proposed approach is demonstrated via several examples and results are presented. The proposed approach cannot entirely replace the function of encoders. Instead, it can be treated as a backup method which provides a reference solution.