DASL Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== How to Use Darwin OP 2's Computer Vision ======


**Author:** Yu Hang He **Email:** <hey6@unlv.nevada.edu> 
\\
**Date:** Last modified on <11/19/17>
\\
**Keywords:** Darwin OP 2, C++/C Programming, Robotis OP 2, Computer Vision
\\

\\
This tutorial will demonstrate how to use vision processing capability on Darwin OP 2 to detect and track color. Darwin OP 2 have computer vision that can detect multiple colors and return x and y center position of those colors. A second class will move camera to keep track of color based on x and y position. It takes approximately 2 hours to complete. 
\\
===== Motivation and Audience =====

This tutorial's motivation is to demonstrate how to use computer vision on Darwin OP 2. This tutorial assumes the reader has the following background and interests: 

  * Familiar with handling Darwin OP 2
  * Familiar with C++/C programming language
  * Familiar with C++/C codes on Darwin OP 2
  * Interested in computer vision

The rest of this tutorial is presented as follows:
  * [[darwin_color_detection#Color_Detection | Color Detection]]
  * [[darwin_color_detection#Camera_Tracking | Camera Tracking]]
  * [[darwin_color_detection#Demonstration | Demonstration]]
  * [[darwin_color_detection#Final_Words | Final Words]]


===== Color Detection =====

The core of image processing on Darwin OP 2 is ColorFinder class. ColorFinder can find the center of mass of detected color and return its location. There is a [[http://support.robotis.com/en/product/darwin-op/development/tutorials/linux/image_processing.htm | tutorial]] on Robotis' online manual that can provide more insights. 
\\ \\
ColorFinder class requires 5 input color parameters: ''hue, tolerance, minimum saturation, minimum value, gain, and exposure''. To find the most accurate set of values for the color you want to detect, use the color_filtering program on Darwin OP 2.
\\ \\
Use terminal and go to directory ''**/robotisop2/Linux/project/tutorial/color_filtering**''
\\ \\
Type in ''make'' and execute the program ''./color_filtering''.
\\ \\ 
{{ :yuhang:darwin_sprint:make.jpg? 500 |}}
\\ 
Once the program is running, open web browser and type in the address http://192.168.123.1:8080
\\ \\
{{ ::yuhang:darwin_sprint:web.jpg? 500 |}}
\\ 
Now you can adjust the values until Darwin OP 2 properly detect your color.
\\ \\
In this example, I am trying to detect a blue color.
\\ \\
**Before adjustments**
\\ \\
{{ :yuhang:darwin_sprint:yuhang:darwin_sprint:before.jpg?500 |}}
\\
**Afterward**
\\ \\
{{ ::yuhang:darwin_sprint:after.jpg?500 |}}
\\
The adjusted values are:
  * ''Hue: 225''
  * ''Tolerance: 15''
  * ''Minimum Saturation: 45''
  * ''Minimum Value: 0''
  * ''Gain: 0.3''
  * ''Exposure: 50.0''



==== Programming ====

This tutorial will demonstrate how to create a new ColorFinder object and how to use it. Only show parts of code that has been modifed.
\\ \\
''main.cpp'' can be found under directory ''/robotisop2/Linux/project/demo''
\\ 
<code cpp | main.cpp> 
//declare a new object of class ColorFinder
ColorFinder* finder = new ColorFinder();

//you can initiate the object with 5 input color parameters
ColorFinder* green_finder = new ColorFinder(120, 45, 35, 0, 0.3, 40.0);

//There are default values of color you can load from ini file. Example
finder->LoadINISettings(ini, "RED"); 

//To get the position of center of mass of filtered pixels
Point2D pos;
pos = finder->GetPosition(LinuxCamera::GetInstance()->fbuffer->m_HSVFrame);
//first declare an object of class Point2D
//then set it equal to GetPosition function of ColorFinder object

</code>



===== Camera Tracking =====

Camera tracking can be accomplished through BallTracker class. An object of this class receive a Point2D object as input, then internally calculate pan and tilt motor movements in Darwin OP 2's head to keep track of position. There is a [[http://support.robotis.com/en/product/darwin-op/development/tutorials/linux/head_control.htm|tutorial]] on Robotis' online manual that may provide more insight.
\\ \\ 
The goal of this demonstration is for Darwin OP 2 to keep track of center of blue and green color. In this demonstration, I created two ColorFinder objects, green_finder and blue_finder, to find the center of mass of its respective colors. Furthermore, I created two Point2D objects, green and blue, to keep track of position of colors. Finally, I used a third Point2D object, center, to store the average between green and blue position.
==== Programming ====

directory ''**/robotisop2/Linux/project/demo**''
\\
This section will only show parts of code that has been modified.

<code cpp | main.cpp>
//inside main loop
int main() {

    ColorFinder* green_finder = new ColorFinder(120, 45, 35, 0, 0.3, 40.0);
    ColorFinder* blue_finder = new ColorFinder(225, 15, 45, 0, 0.3, 50.0);
    //created 2 new ColorFinder objects to find color green and blue
    
    BallTracker marker_tracker = BallTracker();
    //created a new BallTracker object to use camera tracking
   
//inside while loop
while(1) {

else if(StatusCheck::m_cur_mode == SPRINT) //under a new mode called Sprint
    {
    Point2D green, blue, center;
    //created 3 objects of Point2D class
    
    green = green_finder->GetPosition(LinuxCamera::GetInstance()->fbuffer->m_HSVFrame);
    blue = blue_finder->GetPosition(LinuxCamera::GetInstance()->fbuffer->m_HSVFrame);
    //store position of green and blue color
    
    if(green.X < 0 || blue.X < 0)
        center.X = -1;
    else
        center.X = (green.X + blue.X)/2;
    if(green.Y < 0 || blue.Y < 0)
	center.Y = -1;
    else 
        center.Y = (green.Y + blue.Y)/2;
    //calculate center of blue and green    
        
    marker_tracker.process(center);
    //use camera tracking
    }
</code>
===== Demonstration =====

In this demonstration, I used camera track the center between green and blue colors.
\\
\\
First I used color_filtering to find 5 input parameters for green and blue colors.
\\ 
\\ 
{{ ::yuhang:darwin_sprint:2_color.jpg?500 |}}
\\
Then, followed the step outlined in the program section and implemented it within a new mode.
\\
Here is the result.
{{ youtube>4HAZPNL5eY0?large }} 

===== Final Words =====

This tutorial's objective was to demonstrate how to use camera tracking and color detection on Darwin OP 2.
\\
\\
For questions, clarifications, etc, Email: <hey6@unlv.nevada.edu>