Contactless Control of Robotic Arm Through Human Gestures

In this paper the problem of contactless control of robotic arm through human hand gestures is considered. A new method is proposed, which allows to recognize and track the positions of palm and fingertips of human hand in real time and on the basis of these data to implement the control of the wrist and three fingers of the robotic hand. As an input device for capturing hand gestures a depth sensor is used, which works on the principals of triangulation and structured light. Hand gesture recognition is performed by the gradual procession of each video frame, which represents depth images. In the first phase the position of a random point of the palm is detected. In the second phase hand image is extracted using the detected point of the palm. The positions of the fingertips and the lengths of the fingertips are calculated through the analysis of distances between hand contour points and the random point of the wrist. The tracking of the fingertips is achieved through the usage of k-nearest neighbor’s algorithm. The recognized point of the palm is used to control the wrist of the robotic arm in three-dimensional space. The positions of human hand fingertips and lengths of fingers are used to control the fingers of the robotic arm. The proposed method is tested on the computer model of a robotic arm.

gesture recognition, robotic arm, image processing, depth image, robot control