Wait(DELAY);

} sub open_for_write() {

DeleteFile(TEST_FILE_NAME);
CreateFile(TEST_FILE_NAME,TEST_FILE_SIZE,test_handle);

} task main() { open_for_write(); int init; init = NXTCam_Init(camPort, CAMADDR);initialize NXTCam_SendCommand(camPort, CAMADDR, 'E'); enable tracking +++++++++++++SCANNING++++++++++// int bluecount=1,whitecount=1; s="eyeFrontAng="; s+=NumToStr(eyeFrontAng); TextOut(0,LCD_LINE6,s,false); delay(); //++++++++++detect black landmark++++++++++// RotateMotor(EYE,EYE_POWER,-EYE_LEFT); eyeCurAng=-EYE_LEFT; while (eyeCurAng<EYE_RIGHT) { delay(); ClearScreen(); s="eyeCurAng="; s+=NumToStr(eyeCurAng); WriteLn(test_handle,s); TextOut(0,LCD_LINE7,s,false); NXTCam_SendCommand(camPort, CAMADDR, 'B'); // object tracking NXTCam_SendCommand(camPort, CAMADDR, 'E'); // enable tracking delay(); delay(); delay(); NXTCam_GetBlobs(camPort, CAMADDR, nblobs, color, left, top, right, bottom); //saving details of each blob for (colorCount=0;colorCount<nblobs;colorCount++) { s="color:"; s+=NumToStr(color[colorCount]); s+=" left:"; s+=NumToStr(left[colorCount]); s+=" right:"; s+=NumToStr(right[colorCount]); s+=" top:"; s+=NumToStr(top[colorCount]); s+=" btm:"; s+=NumToStr(bottom[colorCount]); area = (right[colorCount]-left[colorCount])*(bottom[colorCount]-top[colorCount]); s+=" area:"; s+=NumToStr(area); WriteLn(test_handle,s); } delay(); RotateMotor(EYE,EYE_POWER,EYE_TURN_ANGLE); eyeCurAng+=EYE_TURN_ANGLE; }//end while RotateMotor(EYE,EYE_POWER,-EYE_RIGHT); NXTCam_SendCommand(camPort, CAMADDR, 'D'); // disable tracking } </syntaxhighlight> Things to take note: *Perform ''#include'' on '''only''' one of the "nxcCam" files (ie. ''nxtcamlib.nxc'', ''nxtcamlib–default.nxc'' or ''nxtcamlib–common.nxc''), otherwise there will be error on having the "same identifier" in multiple files. *From example tutorial "camtest-v3.nxc", the file it ''#include'' is ''nxtcamlib-default.nxc'' where the ''NXTCam_GetBlobs'' function has 8 arguments. If you ''#include'' the ''nxtcamlib.nxc'' file instead, there are only 7 arguments. *Using a sub() to process the results from the ''NXTCam_GetBlobs'' function will not work. Process the values inside the main loop itself. *Do not disable the camera until the end of the program; camera cannot be enabled again once it is disabled in the runtime of the program. *PosRegSetAng() cannot be used once RotateMotor() is used *Limit of NXT runtime for a program is 10mins. NXT switch off after that length of run time regardless if the program has completed or not Below shows the collected figures from the program. [[File:coordinatesNAreas.jpg 400px]] Explanation: *“color” refers to the colormaps pre-configured in the “NXTCam View software”; there are only 2 in this case where 0 caters to the blue landmark while 1 caters to the white one. *“left”, “right”, “top” and “btm” refer to the coordinates of the blobs. *“area” refer to the area of the corresponding blob. Going Further: *With the area value, I can filter out false blobs which are supposedly small, leaving only the bigger blob which is the actual landmark. *The blobs will become small if the landmarks are too far; the area threshold which filters out the false noisy blobs must make sense with the distance threshold which filters out landmarks that are too far. ====Trial 2==== Design of the robot is as shown in the picture below. The camera and sensor are placed at the same position in the horizontal plane along a straight vertical axis as the left wheel, which will be the pivot wheel when turning. This is specially designed as such to eliminated the need for transformation of the position of the sensor to the wheels, hence the vehicle can be modeled as a point in space easily. 400px400px The robot moves in a straight line for 5 steps. At each step it scans 140 degrees in front of itself and record the distance of any detected blob of the designated colors. If detected and the blob has an area big enough to be considered a real landmark, the distance from it to the ultrasonic sensor is recorded. 'Note': Only up to 4 files can be written in an nxc program. The nxc code for this trial is similar to the one above. Here is the video. (I changed the white obstacle to black) Q4xNY9fbCFA The recorded files are displayed below with slight explanations. These files are recorded in a different trial as the one recorded in the video above. *'blue.dat' 300px *If a true blob is not detected in one scan, that is there is no blue object in the a scan, default value of 255 is written into a line in the file. *If a true blob is detected, the angle and the distance at which it is detected is recorded. *If more than one blob is detected, the pair of angle and distance values are recorded separately from the other pairs in a line. *'white.dat' 200px *Same explanation with the blue blob detection above. *Note that a black object is used instead. *'distFwdAngle.dat' 100px *Each line represent the state of the robot at each step of its journey. *In each line, the value before the space is the change in orientation of the robot, while the value after is a state function of whether the robot move forward or not. Thoughts: *Angles made by the motor are not accurate; the motor does not return to the center after each scan. 'Solution': Try using the inbuilt PID controller to reduce errors. *True blobs are not detected. 'Solution': Adjust AREA_THRES and RIGHT_THRES and LEFT_THRES values OR reduce EYE_TURN_ANGLE value to have more extensive scans OR ascertain a more precise or tolerant colormap for the camera to effectively and correctly track the objects. ====Trial 3==== *Motor angles are accurate with the implementation of the PID controller. The tension in the cables connecting to the port of the ultrasonic sensor and the camera gives an unwanted torque to turn the motor and make its angle inaccurate. The PID controller seeks to gives a counter torque to fix that error. This causes the vibration of the camera and sensor. *A lower EYE_TURN_ANGLE to cover more details in each scan increased the success of detecting obstacles, along with adjustments to the other variables. Here are the results of the data collected in the experiment shown in the video. 200px200px fN0vzS6yzbQ ====Trial 4==== Change of experiment plan. Use only one color for the landmark. The ultrasonic sensor will record ALL the values in each scan, regardless if there is a blob detected. The detected blob will be filtered and given a indication of a true blob at the corresponding angle and distance. These changes are to make collection of data easier for easy transfer to Matlab. In addition, I will use a white background instead as the capturing of blobs based on the designated colormap is not very effective. This picture shows the blobs detected when the same colormap is used to capture images at different angles using a green background. Using a white background and black landmarks might make the difference since they are the extremes of the color spectrum. 400px The final experimental set up is shown in the picture below. 400px The nxc code is show below. 'NOTE': There is no obstacle avoidance implemented for this experiment as the maximum runtime of the NXT is limited (up till 10 minutes). The robot will make a 140 degrees scan of the environment in front of it before taking a step forward. It takes 6 steps at maximum. The 'EYE' in the code refers to the motor, ultrasonic sensor and NXT camera assembly. <syntaxhighlight lang='c'> #include “D:\NXT\NXTCAM\nxtcamlib-default.nxc” hardware information EYE #define CAMADDR 0x02 const byte camPort = IN_1; #define EYE OUT_B #define EYE_POWER 20 #define EYE_LEFT 70 #define EYE_RIGHT 70 #define EYE_TURN_ANGLE 5 #define US S2 #define MAX_POWER 75 #define MAX_ACC 15 #define ORIGIN 0 wheels #define R_MOTOR OUT_C #define L_MOTOR OUT_A #define BOTH_MOTOR OUT_AC #define WHEEL_POWER 50 #define WHEELS_TURN_ANGLE 180move forward by 0.5*5.6*pi #define RIGHT_WHEEL_TURN_ANG 136 delay() #define DELAY 1500 file writing int colorCount=0,area,mid,dist,distFwd=0,angle=0; #define EYEANG_FILE_NAME “eyeAng.csv”store angle of eye when true blob detected at each step #define BLOB_FILE_NAME “blob.csv” #define DISTFWD_ANG_FILE_NAME “distFwdAng.csv”store fwd or not at each step and if angle or not #define LM_DIST_FILE_NAME “LMdist.csv” #define FILE_SIZE 1048 #define DEFAULT 255 byte blob_handle=0,distFwd_ang_handle=0,LMdist_handle=0; #define DIST_THRES 45 only register landmark distance if near enof #define AREA_THRES 1200 #define MAX_AREA_THRES 20000 #define LEFT_THRES 46 #define RIGHT_THRES 130 #define BLOBHEIGHT 60 string sblob,sLMdist,sdistFwdAng; for getBlob() int color[10]; int left[10]; int top[10]; int right[10]; int bottom[10]; int nblobs; global variable long eyeCurAng; int journey=0;journey ++ after every scan and move #define END_OF_JOURNEY 7 experiment steps inline void delay() {

Wait(DELAY);

} sub open_for_write() {

DeleteFile(BLOB_FILE_NAME);
DeleteFile(LM_DIST_FILE_NAME);
DeleteFile(DISTFWD_ANG_FILE_NAME);

CreateFile(BLOB_FILE_NAME,FILE_SIZE,blob_handle);
CreateFile(DISTFWD_ANG_FILE_NAME,FILE_SIZE,distFwd_ang_handle);
CreateFile(LM_DIST_FILE_NAME,FILE_SIZE,LMdist_handle);

} sub fwd() {

RotateMotorPID(BOTH_MOTOR,WHEEL_POWER, WHEELS_TURN_ANGLE, PID_1,PID_5,PID_0);
delay();//give time for corection of angle by PID
distFwd=1;
angle=0;

} task main() {

PosRegSetMax(EYE, MAX_POWER, MAX_ACC);
PosRegEnable (EYE, PID_1, PID_5, PID_0);
SetSensorLowspeed(US);
open_for_write();
int init,blobHeight,detected=0;
init = NXTCam_Init(camPort, CAMADDR);//initialize
NXTCam_SendCommand(camPort, CAMADDR, 'E'); // enable tracking
while(journey<END_OF_JOURNEY)
{
	sdistFwdAng=NumToStr(angle);//record state of robot
	sdistFwdAng+=" ";
	sdistFwdAng+=NumToStr(distFwd);
	WriteLn(distFwd_ang_handle,sdistFwdAng);

	//+++++++++++++++SCANNING++++++++++//
	PosRegSetAngle (EYE, -EYE_LEFT);//set the initial starting angle for the eye of the robot
	delay();//give time for correction of angle by PID
	eyeCurAng=-EYE_LEFT;//save current angle of eye
	detected=0;//reset variables before scanning loop
	while (eyeCurAng<=EYE_RIGHT)//per scan
	{
		dist=SensorUS(US);
		detected=0;//reset variables before scanning the next angle

		NXTCam_SendCommand(camPort, CAMADDR, 'B');  // object tracking
		NXTCam_SendCommand(camPort, CAMADDR, 'E'); // enable tracking
		delay();//give time for stablise blob detection
		NXTCam_GetBlobs(camPort, CAMADDR, nblobs, color, left, top, right, bottom);

		//blobFilter
		//saving details of each blob
		if (nblobs!=0)//if blob of the designated color is detected
		{
			for (colorCount=0;colorCount<nblobs;colorCount++)//each of the same color blob tt are detected
			{
				area = abs(right[colorCount]-left[colorCount])*abs(bottom[colorCount]-top[colorCount]);
				//calculate area of each blob detected in this angle
				mid = (right[colorCount]+left[colorCount])/2;
				//calculate mid point of each blob detected in this angle
				blobHeight = abs(bottom[colorCount]-top[colorCount]);
				//calculate height of each blob

				if (area>AREA_THRES && area<MAX_AREA_THRES//if blob has a big enof area, and not too big due to error in detection
					&& blobHeight>BLOBHEIGHT//if blob is tall enof
					&& mid<RIGHT_THRES && mid>LEFT_THRES//if blob lies in the middle of the camera view
					&& dist < DIST_THRES//if blob detected is not too far to be invalid
					&& detected==0) //take in only one blob that pass the filters
				{
					sblob=NumToStr(2);
					Write(blob_handle,sblob);//save value 2 for a true blob deteced in this particular angle
					detected++;
				}

			}//endfor
		}//endif

	if (detected==0)//if at this angle no blobs recognised, value is 0
	{
		sblob=NumToStr(1);
		Write(blob_handle,sblob);//save value 1 for no true blob deteced in this particular angle
	}

	sLMdist=NumToStr(dist);
	Write(LMdist_handle,sLMdist);//save distance detected at this particular angle

	PosRegAddAngle(EYE, EYE_TURN_ANGLE);
	delay();
	eyeCurAng+=EYE_TURN_ANGLE;

	}//end while

	sblob="";
	WriteLn(blob_handle,sblob);//after full 140 degrees scan completed, goes to next line

	sLMdist="";
	WriteLn(LMdist_handle,sLMdist);//after full 140 degrees scan completed, goes to next line

	//+++++++++++end of 1 SCAN+++++++++++//
	PosRegSetAngle (EYE, ORIGIN);//return to front
	delay();//give time for correction of angle by PID

	//++++++obstacle avoidance+++++++++//

	fwd();//move forward after full 140 degrees scan
	journey++;
}//endwhile

NXTCam_SendCommand(camPort, CAMADDR, 'D'); disable tracking } </syntaxhighlight> This is the full video at normal speed. YjQGph-r7jo This is a compilation of the 'blob' and 'LMdist' data collected. 800px *Each line represented the 29 data of all the scans done at each move step. *The last line is incomplete as it has reached the maximum run time of the NXT. *Detected blobs and its corresponding distances detected is highlighted in yellow. *There are no blobs detected on the right side for the 1st few steps because the blobs are too far and thus filtered out. 'Thoughts': *Implementation SLAM is possible as we can see the detected values are in clusters and they are shifting to the side as the robot moves forward. The matlab code can be found here.'GITHUB' The result of the code is shown below. 400px *The black dots represent the true position of the obstacles. *The '*' represents the position of the obstacles detected. *The green line is the true route of the robot by odometry input data. *The red line is the estimated route by going through the EKF. 'Thoughts': *Outlier obstacle detected. *EKF implemented is fairly accurate for this number of steps. *Newly detected obstacles at each step is correctly matched to the nearby obstacles. ====Trial 5==== I tested out the algorithm on another experimental setup as shown below. 400px With an adjustment to the minimum distance threshold between landmarks to be classified as the same landmark, the results are show below. *min_dist=11 400px *min_dist=10 400px *min_dist=8,9 400px *min_dist=7 400px *min_dist=6 400px Thoughts: *The number of observed landmarks increased as the min_dist threshold value decreases. This is expected as observed landmarks whose distance in between them is initially well under the threshold value, is now above and are thus considered distinct landmarks. *At a higher threshold value for min_dist, the number of observed landmarks fit the experimental setup but there is a high possibility of wrong association of true distinct landmarks as the same observed landmark. *Inaccuracy is also due to the limitations of the equipment. Below shows the observed landmarks at the very first scan of the experiment. One of the observed landmark is an abnormally circled in green. It may be due to overlap between the nearer landmark circled in red and the further one circled in blue during scanning. Adjustments need to the various threshold value need to be made to improve the results 400px ===Going Further=== Adjustments to values like RIGHT_THRES, LEFT_THRES, AREA_THRES and EYE_TURN_ANGLE will improve the results. The optimum result will require a good balance of these values.