User Tools

Site Tools


robotic_manipulators_ibvs

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
robotic_manipulators_ibvs [2016/08/08 18:59] – [FlyCapture Library installation guide] joaomatosrobotic_manipulators_ibvs [2016/10/23 20:16] (current) dwallace
Line 1: Line 1:
 ===== Image Based Visual Servoing ===== ===== Image Based Visual Servoing =====
  
- This tutorial will show how to configure and program the Dynamixels servos in C++ using Visual Studio and the library provided by Dynamixel.Also , will show how to configure and program the Flea3 camera in C++ using Visual Studio and the library FlyCapture.+**Author:** Joao Matos Email: <[email protected]> <!-- replace with your email address --> 
 +\\ 
 +**Date:** Last modified on 6/8/2016 
 +\\ 
 +**Keywords:** Visual Servoing , Dynamixel SDK, Flea3 Camera, FlyCapture 
 +\\ 
 + 
 + This tutorial will show how to configure and program the Dynamixels servos in Cpp using Visual Studio and the library provided by Dynamixel.Also , will show how to configure and program the Flea3 camera in Cpp using Visual Studio and the library FlyCapture.
      
  
Line 32: Line 39:
  
    *Download Dynamixel SDK in the [[https://github.com/ROBOTIS-GIT/DynamixelSDK/tree/master/c%2B%2B|github repository]]    *Download Dynamixel SDK in the [[https://github.com/ROBOTIS-GIT/DynamixelSDK/tree/master/c%2B%2B|github repository]]
-   *Go to the subfolder: DynamixelSDK-master\c++\build\win64 and open the dxl_x64_cpp solution. Build the solution in Release and x64.  (this will create the dll in the output folder to be included in the envinment variables).+   *Go to the subfolder: DynamixelSDK-master\cpp\build\win64 and open the dxl_x64_cpp solution. Build the solution in Release and x64.  (this will create the dll in the output folder to be included in the envinment variables).
 {{ ::ii0.jpg?direct |}} {{ ::ii0.jpg?direct |}}
  
Line 40: Line 47:
  
  
-   *Under C++ > general > additional include directories include the path : DynamixelSDK-master\c++\include ( depend on where you saved the github repository. +   *Under Cpp > general > additional include directories include the path : DynamixelSDK-master\cpp\include ( depend on where you saved the github repository. 
-   *Under Linker > general > additional include directories include the path: DynamixelSDK-master\c++\build\win64\output ( depend on where you saved the github repository.+   *Under Linker > general > additional include directories include the path: DynamixelSDK-master\cpp\build\win64\output ( depend on where you saved the github repository.
    *Under Linker > input > additional dependencies add the library dxl_x64_cpp.lib    *Under Linker > input > additional dependencies add the library dxl_x64_cpp.lib
 {{ ::ii2.jpg?direct |}} {{ ::ii2.jpg?direct |}}
Line 51: Line 58:
  Click in environment variables and under system variables find the variable path and click in edit  Click in environment variables and under system variables find the variable path and click in edit
    
- Include the path :  DynamixelSDK-master\c++\build\win64\output.+ Include the path :  DynamixelSDK-master\cpp\build\win64\output.
  
 {{ ::ii4.jpg?direct |}} {{ ::ii4.jpg?direct |}}
Line 61: Line 68:
  
   * Download the [[https://www.ptgrey.com/flycapture-sdk|FlyCapture SDK]]   * Download the [[https://www.ptgrey.com/flycapture-sdk|FlyCapture SDK]]
-  * Install the SDK and open the FlyCapture software. Test and see if your Camera is recognized by your PC (REQUIRED USB 3.0). If it's everything OK you can continue the tutorial and include the library to work with C+++  * Install the SDK and open the FlyCapture software. Test and see if your Camera is recognized by your PC (REQUIRED USB 3.0). If it's everything OK you can continue the tutorial and include the library to work with Cpp
   * Open a solution in the Visual Studio where you already have OpenCV and Dynamixel SDK included.   * Open a solution in the Visual Studio where you already have OpenCV and Dynamixel SDK included.
   * Right click the solution in the solution explorer and go to properties.   * Right click the solution in the solution explorer and go to properties.
-  * Under C++ > general > additional include directories include the directory: Program Files\Point Grey\include+  * Under Cpp > general > additional include directories include the directory: Program Files\Point Grey\include
   * Under Linker > general > additional library directories include the directory: Program Files\Point Grey\lib64\vs2015   * Under Linker > general > additional library directories include the directory: Program Files\Point Grey\lib64\vs2015
   * Under Link > input > additional dependencies include the library: FlyCapture2d_v140.lib   * Under Link > input > additional dependencies include the library: FlyCapture2d_v140.lib
Line 73: Line 80:
 ===== How to use Dynamixel SDK API's ===== ===== How to use Dynamixel SDK API's =====
  
- The control of the dynamixel servos in C++ for the most applications is very straight forward. You will only read / write information through the serial port , and the information depends on the address that you put in the API.+ The control of the dynamixel servos in Cpp for the most applications is very straight forward. You will only read / write information through the serial port , and the information depends on the address that you put in the API.
  
  Depending on your Dynamixel model , the address for certain properties can be different , so check before the address table of your dynamixel. For example , using the model[[http://support.robotis.com/en/product/dynamixel/mx_series/mx-64.htm|mx-64 ]] this how the address table looks like:  Depending on your Dynamixel model , the address for certain properties can be different , so check before the address table of your dynamixel. For example , using the model[[http://support.robotis.com/en/product/dynamixel/mx_series/mx-64.htm|mx-64 ]] this how the address table looks like:
Line 221: Line 228:
  This will open and configure the port to communicate with the USB2Dynamixel .  This will open and configure the port to communicate with the USB2Dynamixel .
  
- To read something (get information) from any dynamixel servo the API is:+** To read something (get information) from any dynamixel servo the API is:**
  
 <Code> <Code>
Line 234: Line 241:
 After the arrow we will define if we want to read or write. If we want to read , we use "read2ByteTxRx" after the packetHandler-> arrow. The first argument is already defined in our startup , the second argument is the dynamixel ID ( you can see which servo correspond to which id in the Dynamixel Wizard) , on this case we are reading an information from the dynamixel servo ID1. The third argument is what you want to read , note that the variable "ADDR_MX_PRESENT_POSITION" is defined as 36 , if we check on the address table from the dynamixel , the address 36 correspond to present position , so we are reading the present position from dynamixel ID1. The fourth argument will hold the result from the reading - the present position - and the last argument will hold an error information (depending on its value we can know if the reading was successful or not). After the arrow we will define if we want to read or write. If we want to read , we use "read2ByteTxRx" after the packetHandler-> arrow. The first argument is already defined in our startup , the second argument is the dynamixel ID ( you can see which servo correspond to which id in the Dynamixel Wizard) , on this case we are reading an information from the dynamixel servo ID1. The third argument is what you want to read , note that the variable "ADDR_MX_PRESENT_POSITION" is defined as 36 , if we check on the address table from the dynamixel , the address 36 correspond to present position , so we are reading the present position from dynamixel ID1. The fourth argument will hold the result from the reading - the present position - and the last argument will hold an error information (depending on its value we can know if the reading was successful or not).
  
- If we want to write something (send information) the idea is very similar to the reading:+** If we want to write something (send information) the idea is very similar to the reading:**
  
 <Code> <Code>
Line 329: Line 336:
  
 This code will create a Mat variable called coloredimage , using RGB channel , to be used in the OpenCV functions. This code will create a Mat variable called coloredimage , using RGB channel , to be used in the OpenCV functions.
 +
 +
 +
 +----
 +===== Integrating vision and control =====
 +
 + The final part of this tutorial is to integrate the dynamixel control with the vision provided by the Flea3 camera. Is important that you have all the libraries working in the same Visual Studio Solution.
 + There are many ways to do the Visual Servoing , and it's up to the programmer decide how to integrate both. 
 +
 + My first code to start to perform the Peg-in-Hole follows the pseudo-algorithm:
 +
 +  - Apply circle detection function to detect hole
 +  - Find the biggest circle detected
 +  - Get radius and center point of the biggest circle detected
 +  - Calculate the distance from the hole to a desired setpoint
 +  - Based on the distance error , move the manipulator up/down/right/left
 +  - if the radius detected is bigger than a threshold the peg is in the hole and must release the gripper
 +  - if not cycle again.
 +
 +The idea here is to make the UAV to hover over hole , and make it go down slowly. As the UAV is moving down next to the hole , the arm will keep the peg centered into the hole . When the radius detected is bigger than a threshold , the camera is very near to the hole and peg should be inside the hole this time , so the gripper must release the peg and the task is done.
 +
 + This is a very simple control algorithm and it will be used to do the preliminary tests with the MM-UAV. We need to see how many disturbance will be created by the arm movement , and how to program a more robust hover control to the UAV in order to make it hold a position next to the hole while moving down.
 +
 + In my code , the angles to make the moving movement are calculated in order to keep the end effector always in the vertical position , because if the end effector is a little bit offset from the vertical position , the peg will not enter in the hole properly , causing a big disturbance in the UAV and this can cause a crash.
robotic_manipulators_ibvs.1470707986.txt.gz · Last modified: 2016/08/08 18:59 by joaomatos