DASL Wiki

User Tools

Site Tools

Trace: • 3d_modeling_using_2d_slices • tutorial_ii • machines_and_mechanisms_tutorial2 • new_layout
drexel_darwin_opencv

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
drexel_darwin_opencv [2016/11/01 15:07] – created dwallacedrexel_darwin_opencv [2016/11/06 19:20] (current) – dwallace
Line 1: Line 1:
 ====== OpenCV ====== ====== OpenCV ======
  
-==Motivation==+===== Motivation ===== 
 Since I will be working on the vision aspect of this Darwin project, I will be engaging OpenCV to process the images received from the camera.  Since I will be working on the vision aspect of this Darwin project, I will be engaging OpenCV to process the images received from the camera. 
  
-OpenCV are libraries of files that languages like C/C++ and Python can use as long as their source codes have the link to the directory of OpenCV. Hence, compare to Matlab which is a standalone software that can also process images, OpenCV is more attractive a choice to outsource the image processing to from Darwin because it will take up less computation memory.+OpenCV are libraries of files that languages like C/Cpp and Python can use as long as their source codes have the link to the directory of OpenCV. Hence, compare to Matlab which is a standalone software that can also process images, OpenCV is more attractive a choice to outsource the image processing to from Darwin because it will take up less computation memory.
  
 I will want to be able to obtain the coordinates of each rung of the ladder which Darwin can use to project its next step in its ascend of the ladder. I will want to be able to obtain the coordinates of each rung of the ladder which Darwin can use to project its next step in its ascend of the ladder.
  
-==Setting up OpenCV== +===== Setting up OpenCL ===== 
-I will be using Visual C++ 2010 Express as my platform for practicing OpenCV. + 
 +I will be using Visual Cpp 2010 Express as my platform for practicing OpenCV. 
  
-I refer to here[https://www.youtube.com/watch?v=2i2bt-YSlYQ] for the installation of the relevant software application and this website[http://www.18f4550.com/OpenCV_and_Emgu_CV/OpenCV_tut_1/OpenCV_tut_1.html], which is related to the previous, to practise and understand some of the codes from the OpenCV library.+I refer to [[https://www.youtube.com/watch?v=2i2bt-YSlYQ|here]] for the installation of the relevant software application and [[http://www.18f4550.com/OpenCV_and_Emgu_CV/OpenCV_tut_1/OpenCV_tut_1.html|this tutorial]], which is related to the previous, to practise and understand some of the codes from the OpenCV library.
  
 There were some error with using OpenCV2.1 and OpenCV2.2 in this case. There were some error with using OpenCV2.1 and OpenCV2.2 in this case.
-*Here[http://answers.opencv.org/question/2582/the-application-is-unable-to-start-correctly/] states that OpenCV2.1 do not work with VC++ 2010. +  * [[http://answers.opencv.org/question/2582/the-application-is-unable-to-start-correctly/|Here]] states that OpenCV2.1 do not work with VCpp 2010. 
-*Here[http://blog.qinyujie.net/2011/01/15/440/] states that there is a bug in OpenCV2.2 that is preventing webcams from working with this version of OpenCV. However, mine still does not work after fixing this bug.+  * [[http://blog.qinyujie.net/2011/01/15/440/|Here]] states that there is a bug in OpenCV2.2 that is preventing webcams from working with this version of OpenCV. However, mine still does not work after fixing this bug.
  
-I have changed to use OpenCV2.4.3 instead. I followed the tutorial here[http://stackoverflow.com/questions/10901905/installing-opencv-2-4-3-in-visual-c-2010-express] to install it properly with VC++ 2010. And I got the program code to run!+I have changed to use OpenCV2.4.3 instead. I followed the tutorial [[http://stackoverflow.com/questions/10901905/installing-opencv-2-4-3-in-visual-c-2010-express|here]] to install it properly with VCpp 2010. And I got the program code to run!
  
-[[File:redBallTracking.jpg|400px]]+{{dylanw:redBallTracking.jpg}}\\ 
  
-'''NOTE''': The OpenCV functions used in this sample code are for C programming, which is not really applicable to my project on Darwin as we will be mainly using C++. For C++, OpenCV uses a different API.+**NOTE**: The OpenCV functions used in this sample code are for C programming, which is not really applicable to my project on Darwin as we will be mainly using Cpp. For Cpp, OpenCV uses a different API.
  
 I have also prepared a MS Power Point presentation for a step by step installation of OpenCV. I have also prepared a MS Power Point presentation for a step by step installation of OpenCV.
-[[:File:OpenCV Tutorial.pptx]] 
  
-==Rectangle Recognition Code== +{{dylanw:opencv_tutorial.pptx}}\\ 
-This[https://github.com/mr1789/darwindasl/blob/master/ladderRungDetection.cpp] is my code to recognize rectangles and output the coordinates of the center of them. +
  
-I will explain the main functions that are critical to make this possible. Refer to here[http://docs.opencv.org/modules/refman.html] for the documentation of the C++ API of OpenCV. +==== Rectangle Recognition Code ====
  
-You may need to go here[http://docs.opencv.org/doc/tutorials/tutorials.html] for a orientation of the OpenCV libraries and built-in functions.+[[https://github.com/mr1789/darwindasl/blob/master/ladderRungDetection.cpp|This]] is my code to recognize rectangles and output the coordinates of the center of them.  
 + 
 +I will explain the main functions that are critical to make this possible. Refer to [[http://docs.opencv.org/modules/refman.html|here]] for the documentation of the Cpp API of OpenCV.  
 + 
 +You may need to go [[http://docs.opencv.org/doc/tutorials/tutorials.html|here]] for a orientation of the OpenCV libraries and built-in functions.
  
 Here are some of the critical parts of the codes: Here are some of the critical parts of the codes:
-<syntaxhighlight lang="cpp">+ 
 +<code c++>
 VideoCapture cap(0); // open the default camera VideoCapture cap(0); // open the default camera
 if(!cap.isOpened())  // check if we succeeded if(!cap.isOpened())  // check if we succeeded
Line 41: Line 46:
 while(1){ while(1){
 cap >> original;  //constantly obtain new frame from camera cap >> original;  //constantly obtain new frame from camera
-</syntaxhighlight> +
-<syntaxhighlight lang="cpp">+
 Mat original, pyr, timg, gray;//declare Mat objects that can be used to store image datas Mat original, pyr, timg, gray;//declare Mat objects that can be used to store image datas
 ... ...
Line 48: Line 52:
 pyrDown(original, pyr, Size(original.cols/2, original.rows/2)); pyrDown(original, pyr, Size(original.cols/2, original.rows/2));
 pyrUp(pyr, timg, original.size()); pyrUp(pyr, timg, original.size());
-</syntaxhighlight> +</code> 
-'''Note''': ''pyrDown'' and ''pyrUp'' can operate on both gray(single channel) and colored(3 channels of Blue Green and Red in the standard BGR order used in OpenCV) images. +**Note**: **pyrDown** and **pyrUp** can operate on both gray(single channel) and colored(3 channels of Blue Green and Red in the standard BGR order used in OpenCV) images. 
-<syntaxhighlight lang="cpp">+ 
 +<code c++>
 inRange(timg,// input timg as the image to be processed inRange(timg,// input timg as the image to be processed
  Scalar(0,  0,  0),// min filtering value (if color is greater than or equal to this)  Scalar(0,  0,  0),// min filtering value (if color is greater than or equal to this)
  Scalar(90,90,90),// max filtering value (if color is less than this)  Scalar(90,90,90),// max filtering value (if color is less than this)
  inrange);//save filtered image to inrange  inrange);//save filtered image to inrange
-</syntaxhighlight> +</code> 
-'''Note''': Resultant filtered image will be a single channel of a matrix of pixels of value of wither 255(white) or 0 (black).+**Note**: Resultant filtered image will be a single channel of a matrix of pixels of value of wither 255(white) or 0 (black).
  
-Inside ''findSquares'' function: +Inside **findSquares** function: 
-<syntaxhighlight lang="cpp">+ 
 +<code c++>
 Canny(image, gray, 50, 200, 5); Canny(image, gray, 50, 200, 5);
 //Detect edges on the input "image" passed by reference from the main program into this findSquare function //Detect edges on the input "image" passed by reference from the main program into this findSquare function
Line 65: Line 71:
 //with lower threshold of 50 and higher threshold to 200 //with lower threshold of 50 and higher threshold to 200
 //with the aperture size of the kernel at 5 //with the aperture size of the kernel at 5
-</syntaxhighlight> +</code> 
-'''Note''': Input image to ''Canny'' function must be a single channel image. It need not be grayscale. +**Note***: Input image to **Canny** function must be a single channel image. It need not be grayscale. 
-<syntaxhighlight lang="cpp">+ 
 +<code c++>
 dilate(gray, gray, Mat(), Point(-1,-1)); dilate(gray, gray, Mat(), Point(-1,-1));
 //take input image, that is the 1st "gray" //take input image, that is the 1st "gray"
Line 74: Line 81:
 //using the kernel of default aperture size of 3 as defined by "Mat()" //using the kernel of default aperture size of 3 as defined by "Mat()"
 //at default start point specified by "Point(-1,-1)" //at default start point specified by "Point(-1,-1)"
-</syntaxhighlight> +
-<syntaxhighlight lang="cpp">+
 vector<vector<Point> > contours; vector<vector<Point> > contours;
 //declaration of a vector of a vector of X and Y points //declaration of a vector of a vector of X and Y points
Line 85: Line 91:
 //processed image is saved to contours //processed image is saved to contours
 //in the designated mode //in the designated mode
-</syntaxhighlight> +</code> 
-'''Note''': Input image to ''findContours'' function must be a single channel image. It need not be grayscale. +**Note***: Input image to **findContours** function must be a single channel image. It need not be grayscale. 
-<syntaxhighlight lang="cpp">+ 
 +<code c++>
 for( size_t i = 0; i < contours.size(); i++ ) for( size_t i = 0; i < contours.size(); i++ )
 { {
Line 126: Line 133:
     }     }
 } }
-</syntaxhighlight>+</code> 
 Here is a sample run of my code to filter out black rectangles and output the X and Y coordinates of the centers of the squares in the output MS-Prompt window. Here is a sample run of my code to filter out black rectangles and output the X and Y coordinates of the centers of the squares in the output MS-Prompt window.
  
-[[File:squaresCoordinates.jpg]]+{{dylanw:squarescoordinates.jpg}}\\ 
  
 Some food for thought: Some food for thought:
-*Code is supposed to outline black rectangles, but it can also encircle the "negative" areas. +  * Code is supposed to outline black rectangles, but it can also encircle the "negative" areas. 
-*Two or more rectangles may be identified for the same rectangle, resulting in double counting. +  * Two or more rectangles may be identified for the same rectangle, resulting in double counting. 
-*This is after all done using Windows Visual C++ which is not applicable on Darwin. +  * This is after all done using Windows Visual Cpp which is not applicable on Darwin. 
-*The speed of computating the codes and data will need to be looked into.+  * The speed of computating the codes and data will need to be looked into. 
 + 
 +==== Real Time Trial On Ladder(1) ====
  
-==Real Time Trial On Ladder(1)== 
 I improved on the previous code and tested it out on an improvised ladder.  I improved on the previous code and tested it out on an improvised ladder. 
  
-Here[https://github.com/mr1789/darwindasl/blob/master/Ladder.cpp] is my code and here is a video of it.+[[https://github.com/mr1789/darwindasl/blob/master/Ladder.cpp|Here]] is my code and here is a video of it.
  
-{{#ev:youtube|R0gCfSxRoOk}}+{{youtube>R0gCfSxRoOk?large}}\\ 
  
 Video explanation: Video explanation:
-*I created a program that creates trackbars for me to use and adjust the colour to filter out easily. +  * I created a program that creates trackbars for me to use and adjust the colour to filter out easily. 
-*I shined a light to test the effect of different light intensity on the colour filtered out. +  * I shined a light to test the effect of different light intensity on the colour filtered out. 
-*Initially the ladder was right in front of the back ground. I shifted the ladder away from the background such that there was some space between them. +  * Initially the ladder was right in front of the back ground. I shifted the ladder away from the background such that there was some space between them. 
-*During the actual detection of the ladder rungs, it was not so effective and I attributed it to the sides of the ladder. They form groove edges which disfigure the rectangular shape of the spaces in between the ladder rungs which were to be detected. +  * During the actual detection of the ladder rungs, it was not so effective and I attributed it to the sides of the ladder. They form groove edges which disfigure the rectangular shape of the spaces in between the ladder rungs which were to be detected. 
-*I cheated and covered the sides so that smooth rectangles were formed. Detection of the spaces in between the ladder rungs improved. +  * I cheated and covered the sides so that smooth rectangles were formed. Detection of the spaces in between the ladder rungs improved. 
-*I went on to detect the actual ladder rungs rather than the spaces in between them.  +  * I went on to detect the actual ladder rungs rather than the spaces in between them.  
-*However, when I removed the covers that were covering the sides of the ladder, non of the rungs can be detected.+  * However, when I removed the covers that were covering the sides of the ladder, non of the rungs can be detected.
  
 In conclusion, the ladder needs to be modified to allow easier detection of the ladder rungs. This can be done by: In conclusion, the ladder needs to be modified to allow easier detection of the ladder rungs. This can be done by:
-*Making a new ladder that has smooth sides so that the corners of each possibly detectable rectangular shape of the spaces in between the runs or the rungs themselves form a smooth rectangle. +  * Making a new ladder that has smooth sides so that the corners of each possibly detectable rectangular shape of the spaces in between the runs or the rungs themselves form a smooth rectangle. 
-*Colour the rungs a different colour and leave the sides untouched. Filter out the colour of the rungs only +  * Colour the rungs a different colour and leave the sides untouched. Filter out the colour of the rungs only 
-*Create a region of interest in the centre of the camera. The detection algorithm will carry out these central pixels only instead of the whole frame.+  * Create a region of interest in the centre of the camera. The detection algorithm will carry out these central pixels only instead of the whole frame. 
 + 
 +==== Real Time Trial On Ladder(2) ====
  
-==Real Time Trial On Ladder(2)== 
 This algorithm depends on contour points (white circles in the picture at the top right hand corner of the insert below) detected to plot rectangles.  This algorithm depends on contour points (white circles in the picture at the top right hand corner of the insert below) detected to plot rectangles. 
  
 However, as can be seen circled in red, when there is a smooth straight line, the points do not appear, and hence on the ladder rungs which has pretty smooth horizontal edge features, points are hard to come by and a rectangle is hard to detect.   However, as can be seen circled in red, when there is a smooth straight line, the points do not appear, and hence on the ladder rungs which has pretty smooth horizontal edge features, points are hard to come by and a rectangle is hard to detect.  
  
-[[File:contourPointsLadder.jpg]]+{{dylanw:contourpointsladder.jpg}}\\ 
  
 This means that one of the adjustments I made during my previous trial run, which was intended to smoothen the sides of the spaces in between ladder rungs by covering up the sides, should make the ladder rungs more difficult to be detected. However the opposite occur.  This means that one of the adjustments I made during my previous trial run, which was intended to smoothen the sides of the spaces in between ladder rungs by covering up the sides, should make the ladder rungs more difficult to be detected. However the opposite occur. 
Line 175: Line 185:
 (I made the left cover slanted to achieve more points along vertical and prove my point) (I made the left cover slanted to achieve more points along vertical and prove my point)
  
-[[File:contourPointsCoveredLadder.jpg|400px]]+{{dylanw:contourpointscoveredladder.jpg}}\\ 
  
 When the ladder is tilted. the detection of rectangles improved greatly (detecting the ladder rungs instead of the spaces in between ladder rungs now by using black colored covers).  When the ladder is tilted. the detection of rectangles improved greatly (detecting the ladder rungs instead of the spaces in between ladder rungs now by using black colored covers). 
Line 183: Line 193:
 Below is a picture of it. Below is a picture of it.
  
-[[File:contourPointsTiltedLadder.jpg|400px]]+{{dylanw:contourpointstiltedladder.jpg}}\\ 
  
-This algorithm which uses contour points and the ''approxPolyDP()'' function is not effective to detect lines which are perfectly vertical or horizontal to the camera, like the ladder rungs. +This algorithm which uses contour points and the **approxPolyDP()** function is not effective to detect lines which are perfectly vertical or horizontal to the camera, like the ladder rungs. 
  
 It is thus not effective to detect the ladder rungs. It is thus not effective to detect the ladder rungs.
  
-==Real Time Trial On Ladder(3)== +==== Real Time Trial On Ladder(3) ==== 
-I found out the best way is to perform canny edge detector on a grayscale image, and then the ''findContours()'' function, rather than filtering out the colour of interest to work on. + 
-<syntaxhighlight lang="cpp">+I found out the best way is to perform canny edge detector on a grayscale image, and then the **findContours()** function, rather than filtering out the colour of interest to work on. 
 + 
 +<code c++>
 ... ...
 ... ...
Line 199: Line 211:
 ... ...
 ... ...
-</syntaxhighlight>+</code>
  
 Here are some pictures. Here are some pictures.
  
-[[File:ladderRungCanny.jpg|400px]][[File:ladderRungSpaceCanny.jpg|400px]]+{{dylanw:ladderrungcanny.jpg}}\\  
 +{{dylanw:ladderrungspacecanny.jpg}}\\ 
drexel_darwin_opencv.1478038036.txt.gz · Last modified: by dwallace