This is an old revision of the document!
Table of Contents
Getting Image and Depth Data from Kinect V2 into OpenVR
Last Date Modified: <04/23/18>
Motivation and Audience
This tutorial's motivation is to show readers how to use a Microsoft Kinect V2 to get image and depth data into the HTC Vive's HMD. Readers of this tutorial are assumed to have the following background and interests:
*Knowledge and experience with OpenVR API
*Knowledge and experience with ROS 1 & 2
*Knowledge and experience with CPP
The rest of this tutorial is presented as follows:
Parts Lists and Sources
These are the parts and their sources needed to complete this tutorial.
| Part Name/Description | Vendor | Vendor URL | Qty | Notes |
|---|---|---|---|---|
| HTC Vive Kit | Vive or Fry's | https://www.vive.com/us/product/ or http://www.frys.com/product/8987273 | 1 | The kit will come with all the necessary hardware. 1 Vive HMD (head mounted display), 2 lighthouse base stations, 2 controllers, and the cables and power sources needed. |
| A computer with at least a GTX 1060 or equivalent | Amazon | https://www.amazon.com/ | 1 | Please reference the recommended computer specs for any computer the reader plans to use. The lab however already has a computer specifically for research with the vive. |
| Tripod mounts for lighthouse base stations | Amazon or build in-house | https://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords=tripod+for+camera | 2 | Since the base stations have a standard camera mounting hole on the bottom any standard camera tripod should work. You can also use scrap pieces of 2×4 wood to create your own mount for them. |
| A computer with Ubuntu 16.04 | Amazon | https://www.amazon.com/ | 1 | The Alienware 17 laptop in the lab should is dual boot and should work well |
| Kinect for Xbox One | Amazon | link | 1 | You will also need to use a converter plate to mount it on HUBO and use a converter to interface with a computer |
| Kinect Adapter for Windows 10 | Amazon | link | 1 | You will need a usb 3.0 port to interface with the converter |
| Plate to Mount Kinect on HUBO | 3D print | No link | 1 | You will need to 3d print a converter to mount the Kinect on Hubo's upper body |
Installing OpenCV and Kinect V2 package
First, you will need to install the following packages on the computer with Ubuntu 16.04 installed.
Note: Though the installation uses Ubuntu 14.04 the 16.04 version will work just fine
OpenVR Code
You will now need to
Importing Video Feed into HMD
First introduce how this tutorial will show the reader how to get image and depth data into the openVR exe file using pre-existing package for ros, and opencv API.
First go through the kinect computer and setting it up. Going through the instalation as well as the calibration of the camera.
Then go into how to create the image wanted with opencv on the camera computer.
Then how to use the ros package to send the stream over the network with http
Finally, talk about how to get the stream from the browser into the exe file using the openvr package