nxt_ar_toolkit
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
nxt_ar_toolkit [2016/08/02 19:39] – [Optional: Step 4: Debugging Marker Recognition] dwallace | nxt_ar_toolkit [2017/07/17 03:13] (current) – [Step 8: Using Multiple Markers for Localization and Navigation] dwallace | ||
---|---|---|---|
Line 3: | Line 3: | ||
**Author:** Dylan Wallace Email: < | **Author:** Dylan Wallace Email: < | ||
\\ | \\ | ||
- | **Date:** Last modified on 08/02/16 | + | **Date:** Last modified on 08/28/16 |
\\ | \\ | ||
**Keywords: | **Keywords: | ||
\\ | \\ | ||
- | {{ :sampleimageforwritingtutorial.jpg?200 |}} | + | {{ :dylanw: |
\\ | \\ | ||
- | The photo above depicts | + | The photo above depicts |
\\ | \\ | ||
===== Motivation and Audience ===== | ===== Motivation and Audience ===== | ||
Line 19: | Line 19: | ||
* Know how to use basic Linux and Windows commands | * Know how to use basic Linux and Windows commands | ||
\\ | \\ | ||
- | * Perhaps also know how to program in C/C++ | + | * Perhaps also know how to program in C/Cpp, especially with Visual Studio. |
\\ | \\ | ||
- | * Perhaps additional background needed may include experience with other applications | + | * Perhaps additional background needed may include experience with other frameworks |
\\ | \\ | ||
* This tutorial may also attract readers who are interested in path planning, computer vision, NXT applications, | * This tutorial may also attract readers who are interested in path planning, computer vision, NXT applications, | ||
Line 44: | Line 44: | ||
==== Step 3: Obtain Missing DLLs ==== | ==== Step 3: Obtain Missing DLLs ==== | ||
- | If you are running on a newer Windows installation, | + | If you are running on a newer Windows installation, |
===== Using the ARToolkit ===== | ===== Using the ARToolkit ===== | ||
Line 94: | Line 94: | ||
**Orange/ | **Orange/ | ||
+ | ==== Step 5: Creating a New Project ==== | ||
+ | |||
+ | Once you have the example program simpleLite working well with marker recognition and tracking, you can move onto creating your own project. For developing your own projects using the ARToolkit, I recommend using Visual Studio 2013 or higher. Visual Studio has great support for the ARToolkit SDK, and all of their example projects have been provided as Visual Studio projects to modify or use as reference. | ||
+ | |||
+ | **Note:** With recent updates to ARToolkit, it can be very difficult to create a new project in Visual Studio for ARToolkit. If you run into these issues, I recommend doing your development within and already-existing project within the provided ARToolkit5 Solution. This will make the process much easier to debug, and take away the worries of creating and maintaining your own project. | ||
+ | |||
+ | To begin with creating a new project, simply open up Visual Studio and create an empty Visual Cpp project. Once you have created the empty project, you will need to add a .c file to the project. I named my file coordinate_tracking.c, | ||
+ | |||
+ | Once you have created the project, you will need to add the include and library directories to your project. Do this by right-clicking the project, and going to Properties. Under properties, look for the VCpp Directories tab. First click on the drop-down for include directories, | ||
+ | |||
+ | Finally, you need to copy over some data from the ARToolkit5 bin folder. First, copy the Data folder over to your project folder. Next, copy DSVL.dll, DSVLd.dll, ARvideo.dll, | ||
+ | |||
+ | Once you have all of this configured, you are ready to create your first program with the ARToolkit SDK. | ||
+ | |||
+ | ==== Step 6: Modifying the Simple Program to Track Coordinates ==== | ||
+ | |||
+ | Now that you have successfully setup, tested, and even created a new project with the ARToolkit SDK, you are finally ready to create custom code with the ARToolkit SDK. However, instead of creating all of the code from scratch, it is good to build off of the simpleTest.c program, provided with the ARToolkit SDK. | ||
+ | |||
+ | For reference on how the simpleTest.c program works wth the ARToolkit SDK, refer to [[https:// | ||
+ | |||
+ | **Adding Variables**\\ | ||
+ | Add these two variable declarations after line 31 of the simpleTest.c program. These lines will allow us to print the X and Y coordinates to the screen during processing. | ||
+ | <code c++> | ||
+ | 32 char xValue[256]; | ||
+ | 33 char yValue[256]; | ||
+ | </ | ||
+ | |||
+ | It is also necessary to add some variables to the mainLoop function. Add these variable declarations after line 140. Keep in mind, xReal and yReal are dependent on what the camera displays in your view, and its real-world size. | ||
+ | <code c++> | ||
+ | 141 double xCoord; | ||
+ | 142 double yCoord; | ||
+ | 143 double xReal = 38.5; | ||
+ | 144 double yReal = 26.5; | ||
+ | </ | ||
+ | |||
+ | **Flipping Video**\\ | ||
+ | Since the camera setup is one that is looking down onto the table from a reversed angle, we need to flip the video both horizontally and vertically. We can achieve this by adding 15 lines of code in the main function, after line 54. | ||
+ | <code c++> | ||
+ | 55 if (flipMode & AR_GL_FLIP_V) | ||
+ | 56 flipMode = flipMode & AR_GL_FLIP_H; | ||
+ | 57 else | ||
+ | 58 flipMode = flipMode | AR_GL_FLIP_V; | ||
+ | 59 | ||
+ | 60 argViewportSetFlipMode(vp, | ||
+ | 61 | ||
+ | 62 if (flipMode & AR_GL_FLIP_H) | ||
+ | 63 flipMode = flipMode & AR_GL_FLIP_V; | ||
+ | 64 else | ||
+ | 65 flipMode = flipMode | AR_GL_FLIP_H; | ||
+ | 66 | ||
+ | 67 argViewportSetFlipMode(vp, | ||
+ | </ | ||
+ | |||
+ | **Removing Current Output**\\ | ||
+ | Befoire adding new output of the X and Y coordinates, | ||
+ | <code c++> | ||
+ | 189 if (count % 60 == 0) { | ||
+ | 190 // | ||
+ | 191 | ||
+ | 192 } | ||
+ | 193 count++; | ||
+ | 194 // | ||
+ | 195 // | ||
+ | </ | ||
+ | |||
+ | For the error values, comment out lines 227, 229, and 230. | ||
+ | <code c++> | ||
+ | 227 // | ||
+ | 228 glColor3f(0.0f, | ||
+ | 229 // | ||
+ | 230 // | ||
+ | 231 // | ||
+ | </ | ||
+ | |||
+ | **Adding the X & Y Coordinates**\\ | ||
+ | Now we need to add the data for the X and Y coordinates to the screen. We can achieve this by adding 6 lines of code to the same area we just removed code from. Add these four lines after line 226. | ||
+ | <code c++> | ||
+ | 227 xCoord = markerInfo[k].pos[0]; | ||
+ | 228 yCoord = markerInfo[k].pos[1]; | ||
+ | 229 sprintf(yValue, | ||
+ | 230 sprintf(xValue, | ||
+ | </ | ||
+ | |||
+ | Now add these two lines after line 232. | ||
+ | <code c++> | ||
+ | 233 argDrawStringsByIdealPos(xValue, | ||
+ | 234 argDrawStringsByIdealPos(yValue, | ||
+ | </ | ||
+ | |||
+ | **Changing the Graphics Models**\\ | ||
+ | Now that we are tracking the coordinates well, we just need to change to original 3D cube graphics into a small 3D marker. This can be achieved by simply editing two lines of code, 388 and 389. | ||
+ | <code c++> | ||
+ | 388 glTranslatef(0.0f, | ||
+ | 389 glutSolidCube(10.0); | ||
+ | </ | ||
+ | |||
+ | |||
+ | Now the program should run smoothly and with accurate prediction of the real-world coordinates of the marker. Below is a video showing the coordinate tracking program in action. | ||
+ | |||
+ | {{ youtube> | ||
+ | |||
+ | ==== Step 7: Tracking Multiple Markers ==== | ||
+ | |||
+ | In order to use the ARToolkit SDK for tracking of objects or for any kind of computer vision application, | ||
+ | |||
+ | **Changing from pattern-detection to barcode-detection** | ||
+ | |||
+ | First you will need to remove the references to the pattern style, because we are no longer using pattern-detection. To do this, comment out the lines in the beginning of the program around line 25 (depending on your code) where it defines the pattern name, and where it initializes the pattern handle. | ||
+ | |||
+ | //# | ||
+ | | ||
+ | // | ||
+ | | ||
+ | Now you will need to add the barcode-detection to the main function. Places these two lines of code right before the function call for argMainLoop(): | ||
+ | |||
+ | arSetPatternDetectionMode(arHandle, | ||
+ | arSetMatrixCodeType(arHandle, | ||
+ | | ||
+ | This will setup the marker-detection into barcode-matrix mode, allowing the ARToolkit to detect these markers instead of the traditional style of marker. | ||
+ | |||
+ | **Tracking Coordinates of Multiple Markers** | ||
+ | |||
+ | In order to consistently track the coordinates of multiple markers, you will need to alter the detection algorithm to associate one barcode ID with one instance of an array for storing your coordinates. To do this, add these lines of code to the main detection loop, after the IDs and CF values are logged to the ARConsole (Please change xCoord and yCoord to arrays that have the size as the amount of markers to track (currently 8 in this example)): | ||
+ | |||
+ | for (h = 0; h < 7; h++) { | ||
+ | | ||
+ | if (markerInfo[j].idMatrix == h) { | ||
+ | yCoord[h] = (yReal / ysize)*(ysize - markerInfo[j].pos[1]); | ||
+ | xCoord[h] = (xReal / xsize)*(markerInfo[j].pos[0]); | ||
+ | } | ||
+ | } | ||
+ | | ||
+ | Now we will be adding to this for loop in order to use our tracked coordinates for localization and navigation within an area. | ||
+ | |||
+ | ==== Step 8: Using Multiple Markers for Localization and Navigation ==== | ||
+ | |||
+ | In order to track the position of a robot or other vehicle using the ARToolkit, we will need to create an algorithm that creates relative coordinates for the robot based off of a known location of a stationary marker. This will allows us to create a kind of coordinate system that is independent of camera position or resolution. In order to do this, we will need to use one barcode marker (in most cases this will be number 0) fixed to a stationary location, and then reference this location in our code to get relative distances. | ||
+ | |||
+ | if ((xCoord[0] != 0) && (yCoord[0] != 0) && (xCoord[h] < 1000) && (yCoord[h] < 1000) && (xCoord[h] > -1000) && (yCoord[h] > -1000)) { | ||
+ | xCoordRel[h] = xCoord[h] - xCoord[0]; | ||
+ | yCoordRel[h] = yCoord[h] - yCoord[0]; | ||
+ | ARLOG(" | ||
+ | sprintf(xValue[h], | ||
+ | sprintf(yValue[h], | ||
+ | argDrawStringsByIdealPos(xValue[h], | ||
+ | argDrawStringsByIdealPos(yValue[h], | ||
+ | } | ||
+ | |||
+ | This if statement will store the positions of these markers into a separate array for relative positions, ensuring that normal tracking is unaffected by this. This will also output those relative distances to the screen using the last 5 lines of code. Since this is located within that same for loop from above, this will keep everything sorted into the proper positions of their array. The conditional statement for this step is checking to make sure that the reference marker is stored, and that the marker it is trying to calculate for is being recognized correctly (hence the over 1000 value). Once you have added this to your code, you should | ||
+ | \\ | ||
+ | {{ youtube> | ||
+ | \\ | ||
===== Final Words ===== | ===== Final Words ===== | ||
- | This tutorial' | + | This tutorial' |
\\ | \\ | ||
\\ | \\ | ||
- | Speculating future work derived from this tutorial, includes | + | Speculating future work derived from this tutorial, includes |
\\ | \\ | ||
\\ | \\ | ||
- | For questions, clarifications, | + | For questions, clarifications, |
nxt_ar_toolkit.1470191945.txt.gz · Last modified: 2016/08/02 19:39 by dwallace