Hybrid Vision Based Reach-to-Grasp Task Planning Method for Trans-Humeral Prostheses

Abstract

This paper proposes a hybrid vision based reachto- grasp task planning method for trans-humeral prostheses exploiting both vision and Electromyography (EMG) signals. The hybrid method mainly consists of 2-1/2D visual servoing module and EMG based module. The visual servoing intends to align the object on to the center of the palm while correcting its orientation. EMG signals extracted from the remaining muscles of the disabled arm due to amputation are used to control the elbow flexion/extension (FE). While using the 2-1/2D visual servoing module, the object reaching algorithm changes the elbow FE angle to reach the palm towards the object of interest. Initially, the EMG based module controls the elbow FE. Once an object is detected, the EMG signals emanating from the arm muscles generates a reach request. This process then activates the visual servoing module to bring the palm towards the object. Since both EMG based module and the visual servoing module are producing elbow FE angles while reaching towards an object, these two modules are integrated to obtain a resultant angle for elbow FE. Experiments are conducted using a simulation environment and a prosthesis to validate the proposed task planning method. The EMG based module is capable of following the natural elbow FE motion. Moreover, the task planning method is capable of driving the prosthesis towards the object with proper orientation.