This is an old revision of the document!
Tutorial 7
On this tutorial you will learn how to apply threshold to track an object by its center of mass using image moments.
I recommend you to type the code on your own to get familiarized with the program language. If you have trouble , the original code is attached bellow ( Running on Visual Studio 2015 + OpenCV 3.1 ) * Check the installation guide to make sure that you linked all the OpenCV modules to your Visual Studio.
Tracking an object using image moments
//the erode element chosen here is a 3x3 piexels rectangle. //Change the Size argument to optimize your thresholding. //dilate with 8x8 size element to make the thresholding object more visible
Mat erodeElement = getStructuringElement(MORPH_RECT, Size(3, 3));
Mat dilateElement = getStructuringElement(MORPH_RECT, Size(8, 8));
Apply erode and dilate erode(image, image, erodeElement); dilate(image, image, dilateElement); } </Code> Again Exactly the same function used on Tutorials 5 and 6. You can read the step by step here. When this function is called it will apply the morphological transformations dilate and erode to optimize the object silhouette on the threshold image. —- <Code Clinenums:84> //Function to track an object using image moments. void TrackObject(Mat threshold, Mat &image) { //We will not use the threshold image directly we will use a copy //if you use the original threshold , when you turn the tracking ON , the window //displaying the threshold image will have an interference. Mat temp; threshold.copyTo(temp); //x and y values for the location of the object int x = 0, y = 0; //Tracking success. bool objectFound = false; //Variables to store the largest area on the threshold image int largest_area = 0; //Initializing two vectors to be used on the findContours function vector<vector<Point> > contours; vector<Vec4i> hierarchy; //findContours function using the output of the threshold operation findContours(temp, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE); </Code> To make the program more readable lets separate the algorithm that will track the object from the main program - making it as a function and only be used when it's called on the main program ( we will create a on/off toggle similar to the morphological transformations use from the previous tutorials ). The arguments will be the column position of the center of mass (x) , the row position of the center o mass (y) , the image after the threshold made by the **"inRange"** function and the original image. To avoid errors lets use a copy of the threshold image on the tracking algorithm , this way you will not get any interference on the window that is showing the threshold image when you toggle on/off the tracking algorithm. We can easily copy the threshold image into other variable using **".copyTo()"**. Lets define a Boolean to say if we the tracking algorithm is running or not. Also a variable that will store the largest contour area will be set to help when filtering the contours found by the **"findContours"** function. Also lets define the vectors that will be used by the **"findContours"** function , same as Tutorials 6 and 4. A vector to store the contour points and a vector to the hierarchy -the hierarchy vector contain as many element as the number of contours found (if we know its size , we know how many contours were found) , and contains information about the image topology . We find all the contours in the threshold image (that was copied to the temp variable ) using the **"findContours"** function . ---- <Code C++ linenums:107> //use moments method to find our filtered object if (contours.size() > 0) { int numObjects = contours.size(); //if number of objects greater than MAX_NUM_OBJECTS we have a noisy filter if (numObjects<MAX_NUM_OBJECTS) { </Code> Now we start the loop that will search for the largest area , and will calculate the center of mass of this area using the image moments. Image moments is analogy to the moments in physics. The 'weight' of each pixels is denoted by its intensity , so if the pixel has 'mass' it will have a center of mass (read more about image moments [[https://www.quora.com/What-exactly-are-moments-in-OpenCV linenums:114> for (int i = 0; i < contours.size(); i) { //Image moments: m00 is the area , m10/m00 is the column of the center of mass //m01/m00 is the row of the center of mass. Moments moment; moment = moments((Mat)contours[i]); double area = moment.m00; //Find the largest contour to be draw on the image //Also filter using a minimum and maximum area value to avoid false positives. //When all the conditions are true we probably have a track able object. if (area>min_area && area<max_area && area>largest_area) { x = moment.m10 / area; //center of mass column y = moment.m01 / area; //center of mass row objectFound = true; largest_area = area; } else objectFound = false; </Code> To find the image moments we can use a OpenCV function for it , but first we need to set a variable that will store those moments. The variable type is "Moments" and lets call the variable moment. We can get the moments from each contour as the loop goes on ('i' increases) using the assign **"moment = moments((Mat)contours[i])"** - which will store the moments of the i-contour on the moment variable. You can check [[http://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html?highlight=drawcontours linenums:140> if we found an object lets track it
if (objectFound == true)
{
putText(image, "Object Found", Point(0, 50), 2, 1, Scalar(0, 255, 0), 2);
circle(image, Point(x, y), 3, Scalar(0, 255, 0), -1, 8, 0);
putText(image, "(" + intToString(x) + "," + intToString(y) + ")" , Point(x, y + 30), 1, 1, Scalar(0, 255, 0), 2);
}
else putText(image, "Searching for object", Point(0, 50), 1, 2, Scalar(0, 0, 255), 2);
}
}
} </Code>
Now if we found an object we want to track its position. So the conditional statement if the objectFound variable is true will be used to print on the screen some useful informations. Lets draw a small circle on the center of mass of the contour and print the value of the center of mass column and row.
The first “putText” function will display on the top of the screen that an object was found. The “circle” function will draw a circle centered in (x,y)- which has stored the value of the center of mass column and row - with radius 3. The second “putText” will display the (x,y) value on the screen , a little bit offset from the center to make the visualization of the center more clear.
If the object found flag is not true , it will display on the top of the screen that the searching is on progress. And this is the end of our tracking algorithm.
<Code C++ linenums:155> int main(int argc, char* argv[]) {
//some boolean variables for different functionality within this //program bool useTrack = false; bool useFeatures = false; char key = 0;
Matrix to store each frame of the webcam feed Mat coloredimage; matrix storage for HSV image
Mat HSV; //matrix storage for binary threshold image Mat threshold; //create slider bars for HSV filtering createTrackbars(); //video capture object to acquire webcam feed VideoCapture capture; //open capture object at location zero (default location for webcam) capture.open(0); //set height and width of capture frame capture.set(CV_CAP_PROP_FRAME_WIDTH, FRAME_WIDTH); capture.set(CV_CAP_PROP_FRAME_HEIGHT, FRAME_HEIGHT);
</Code>
We already have all the algorithm separated into functions , our main program will only be to make sure everything is called in the right time and to create the on/off toggles to show the morphological transformations and the tracking algorithm.
We start defining a set of variables that we will use. Mat variables to store the web camera video , the HSV version of this video and a threshold version after the “inRange” function do the threshold. Also we define the x and y positions of the center of mass ( calculated on the track function ).
Also lets initialize the video capture from the web camera video , and set the video properties using “capture.open(0)“ and “capture.set()“.
<Code C++ linenums:182>
//Until the User press q the loop will run
//Get the image from the webcam -> convert to HSV -> Threshold the image
//using the HSV max and min set on the Trackbar window.
while (key != 'q')
{
//Get the image from the webcam
capture >> coloredimage;
//Convert the frame from BGR (RGB) to HSV
cvtColor(coloredimage, HSV, COLOR_BGR2HSV);
//filter the HSV image using the minimum and maximum values set on the
//Trackbars window using the inRange function.
inRange(HSV, Scalar(iLowH, iLowS, iLowV), Scalar(iHighH, iHighS, iHighV), threshold);
</Code>
The loop will run until the 'q' key is pressed . First thing is to capture a frame from the web cam video and store it on the 'coloredimage' - which is converted after using the “cvtColor” to HSV to be used on the threshold process. The threshold process is done by hand , slinding the bars in the Trackbars window . The maximum and minimum values will be used by the “inRange” function to do the threshold ( same process from Tutorial 6 and 5 ). It will output a binary image stored on the last argument - 'threshold'.
<Code C++ linenums:199>
//If 'm' is pressed it will turn on/off the morphological transformations
//dilate and erode
if (key == 'm')
{
useFeatures = !useFeatures;
}
if (useFeatures)
{
DilateAndErode(threshold);
}
If 't' is pressed it will turn on/off the tracking algorithm First we need to treshold the image by hand , so it will start as false
//After you get a good threshold image you can turn the tracking ON by pressing 't'.
if (key == 't')
{
useTrack = !useTrack;
}
if (useTrack)
{
TrackObject(threshold, coloredimage);
}
</Code>
Here we create the on/off toggles to use the morphological transformations (by pressing 'm') and to run the tracking algorithm (by pressing 't') . Each time 'm' or 't' is pressed the Booleans useTrack and useFeatures will change values from TRUE to FALSE and vice-versa. Every time these values are TRUE , it will enter inside the conditional loop that will call the respective functions , “DilateAndErode()“ and “TrackObject()“.
<Code C++ linenums:226>
//show frames
imshow("Threshold Image", threshold);
imshow("Live Web Cam", coloredimage);
imshow("HSV Image", HSV);
key = waitKey(25);
}
return 0; }
</Code>
Finally we open and display three different windows to show the Live web cam video , the threshold version and the HSV version. The key value is updated using “key = waitKey(25)“. And our program is done !