Discovery Center - Gesture controlled robot

The goal of the project is to equip the Discovery Center of Idaho with their first fully functioning, motion actuated robotics exhibit. This exhibit will feature a robot arm that can be controlled using hand gestures as well as a variety of activities that users can complete based on different skill tiers (beginner, intermediate, and advanced).

Problem Definition
We need to be able to control a robotic arm using hand gestures, tracked through a motion sensory device. In addition, our robot must be able to operate within the confines of it's enclosure such that it is safe for the user to operate and that the robot is not damaged by user input.

Background
Currently, the Discovery Center of Idaho does not have a robotics exhibit to show. This project will allow children and patrons of all ages to learn about robotics as well as interact with a live robot that they can control using only their hand gestures.

Deliverables
 Fully assembled robot arm with leap motion sensor Software program to control robot Documentation on how to operate/ maintain/ troubleshoot the system  

Specifications
 Robot arm must be compact and transportable 2-3 activities that the robot arm can perform based on user skill  Beginner Intermediate</li> Advanced</li> </ul> User must be able to control robot using a Leap Motion controller</li> Activities must be able to be reset easily for the next user</li> Parts used (actuators, microcontrollers, etc.) must be easily replaceable and maintainable</li> Exhibit must be safe for all ages to operate</li> </ul>

Leap Motion
The Leap Motion controller is a motion tracking device that can be used to track hand gestures. The controller is capable of tracking an individual's palm, forearm, and each individual finger joint. In addition, the sensor can track two hands at a time. This controller tracks hand gestures using two different cameras, and is found to perform better when overhead light is at a minimum.



Servo Diagram
This diagram shows the configuration of the servos used to actuate the robot arm.


 * Servo 1 controls the base rotation
 * Servos 2 and 3 control shoulder movement
 * Servos 4 and 5 control elbow movement
 * Servo 6 controls the vertical movement of the wrist
 * Servo 7 controls the wrist rotation
 * Servo 8 controls the end effector



System Diagram 1
This diagram shows the system using a Arbotix micro-controller to control the servos and provide the robot with power.



System Diagram 2
This diagram shows the system using a U2D2 USB converter, which would subvert the micro-controller to control the servos.



System Diagram 3
This diagram shows the system using an XBee receiver for a wireless hand-held controller, replacing the Leap Motion.



XBee wireless controller
In order to learn how the robot responds to certain inputs, we opted to try and control the robot via a handheld remote. We bought the remote from the robot's manufacturer and used the code provided to gain insights into how we could eventually use the Leap Motion controller.



Final Design
For our final system design, we decided it would be best if we used the Arbotix micro-controller since it reduced complexity and freed the U2D2 device to be used to ID other servos.

Final design system diagram:



XBee
We were able to successfully control the robot using the XBee controller and demonstrated this at snapshot #2.



Gripper Modification
Since the size of our initial gripper was inadequate, we designed and 3D-printed a new, wider gripper so that we could grasp larger objects.

Inverse Kinematics functionality
We were able to implement the use of inverse kinematics to move the robot arm using the Leap Motion controller by retrofitting the code from the XBee controller to use the input coordinates from Leap Motion.

Gripper functionality
We were able to control the gripper by tracking the thumb and other digits and then mapping the distance between the digits to a range of servo values.

Additional Documentation
Project Schedule



Meeting Minutes



Presentations



Client Interview