Differences

This shows you the differences between two versions of the page.

--- pegasus_cam_homogoraphy [2019/03/05 03:20] – dwallace
+++ pegasus_cam_homogoraphy [2019/03/08 04:54] (current) – [Networked Video Homography using the Pegasus Cam] dwallace
@@ Line 8: / Line 8: @@
 \\
 This tutorial shows how to implement real-time networked video homography using the Pegasus Cam. The big picture problem is to allow small teams of robots to collaborate using shared visual information. Solving this partially or completely is important because it will allow for teams of robots to collaborate in scenarios such as disaster rescue and cleanup, emergency response, and human-robot collaboration. This tutorial demonstrates how to implement real-time networked video homography using Python OpenCV and the Pegasus Cam. This tutorial takes approximately 1 hour to complete.
-\\
+The source code for the real-time networked homography with the Pegasus Cams can be seen on the [[https://github.com/D-Wazzle/Pegasus-Cam-Homography|Pegasus Cam Github]].
 ===== Motivation and Audience =====
@@ Line 24: / Line 26: @@
 \\
 The rest of this tutorial is presented as follows:
-  * [[pegasus_cam_homographyp#homography_overview|Homography Overview]]
+  * [[pegasus_cam_homogoraphy#homography_overview|Homography Overview]]
-  * [[pegasus_cam_homography#code_overview|Code Overview]]
+  * [[pegasus_cam_homogoraphy#code_overview|Code Overview]]
-  * [[pegasus_cam_homography#demonstration|Demonstration]]
+  * [[pegasus_cam_homogoraphy#demonstration|Demonstration]]
   * Final Words
 ===== Homography Overview =====
-\\
+Homography is the process of meshing two visual planes using key point in the image as 3D reference points. The algorithm for running the real-time homography program is quite simple. First we get the Pegasus Cam streams. Then for every frame in the two streams:
+. Extract keypoints using Difference of Gaussians
+. Extract invariant descriptors using SIFT/SURF
+. Match keypoints between the two images using the descriptors
+. Use RANSAC to create a homography matrix from the matched feature vectors
+. Apply the homography matrix transformation
+However, in order to optimize this program for real-time, it is more desirable to complete steps 1-4 for the first few frames, and then use that homography matrix for the rest of the stream, only applying the transformation for every frame of the two video streams.
+Good examples of homography in common life is creating a panoramic image with your smartphone.
 ===== Code Overview =====
+The code for the real-time homography was implemented in Python. The full code with detailed comments can be seen below.
+<code python>
+# USAGE
+# python realtime_stitching.py
+# import the necessary packages
+from __future__ import print_function
+from pyimagesearch.basicmotiondetector import BasicMotionDetector
+from pyimagesearch.panorama import Stitcher
+from imutils.video import VideoStream
+import numpy as np
+import datetime
+import imutils
+import time
+import cv2
+# initialize the Pegasus Cam video streams and allow them to warmup
+print("[INFO] starting cameras...")
+leftStream = cv2.VideoCapture("http://192.168.50.22:3000/html/cam_pic_new.php?time=9999999999999&pDelay=40000")
+rightStream = cv2.VideoCapture("http://192.168.50.23:3000/html/cam_pic_new.php?time=9999999999999&pDelay=40000")
+time.sleep(2.0)
+# initialize the image stitcher, motion detector, and total
+# number of frames read
+stitcher = Stitcher()
+motion = BasicMotionDetector(minArea=500)
+total = 0
+# loop over frames from the video streams
+while(1):
+	# grab the frames from their respective video streams
+	retl, left = leftStream.read()
+	retr, right = rightStream.read()
+	# resize the frames
+	left = imutils.resize(left, width=512)
+	right = imutils.resize(right, width=512)
+	# stitch the frames together to form the panorama
+	# IMPORTANT: you might have to change this line of code
+	# depending on how your cameras are oriented; frames
+	# should be supplied in left-to-right order
+	result = stitcher.stitch([left, right])
+	# no homograpy could be computed
+	if result is None:
+		print("[INFO] homography could not be computed")
+		break
+	# convert the panorama to grayscale, blur it slightly, update
+	# the motion detector
+	gray = cv2.cvtColor(result, cv2.COLOR_BGR2GRAY)
+	gray = cv2.GaussianBlur(gray, (21, 21), 0)
+	locs = motion.update(gray)
+	# only process the panorama for motion if a nice average has
+	# been built up
+	if total > 32 and len(locs) > 0:
+		# initialize the minimum and maximum (x, y)-coordinates,
+		# respectively
+		(minX, minY) = (np.inf, np.inf)
+		(maxX, maxY) = (-np.inf, -np.inf)
+		# loop over the locations of motion and accumulate the
+		# minimum and maximum locations of the bounding boxes
+		for l in locs:
+			(x, y, w, h) = cv2.boundingRect(l)
+			(minX, maxX) = (min(minX, x), max(maxX, x + w))
+			(minY, maxY) = (min(minY, y), max(maxY, y + h))
+		# draw the bounding box
+		cv2.rectangle(result, (minX, minY), (maxX, maxY),
+			(0, 0, 255), 3)
+	# increment the total number of frames read and draw the
+	# timestamp on the image
+	total += 1
+	timestamp = datetime.datetime.now()
+	ts = timestamp.strftime("%A %d %B %Y %I:%M:%S%p")
+	cv2.putText(result, ts, (10, result.shape[0] - 10),
+		cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255), 1)
+	# show the output images
+	cv2.imshow("Result", result)
+	cv2.imshow("Left Frame", left)
+	cv2.imshow("Right Frame", right)
+	key = cv2.waitKey(1) & 0xFF
+	# if the `q` key was pressed, break from the loop
+	if key == ord("q"):
+		break
+# do a bit of cleanup
+print("[INFO] cleaning up...")
+print(total)
+cv2.destroyAllWindows()
+leftStream.release()
+rightStream.release()
+</code>
 \\
 ===== Demonstration =====
+After implementing the real-time homography code form the previous section, you should get results similar to the video below.
+\\
+{{ youtube>GACdIFXawmM?large }}
 \\