User Tools

Site Tools


akshayjournal

This is an old revision of the document!


Background

My name is Akshay Dave, and I am a third-year undergraduate student doing mechanical engineering. Right now I am in Korea doing research at KAIST's HUBO Lab. This my blog

Week 1: 06/10/19 - 06/17/19

1. Photo of the week

The biggest thing this week was that Hubo lab has a mark forged 3D Printer. it can print with carbon fiber. The reason it made me super excited was that I never used a printer like this before. I always wanted to learn how to use it. Also, Dr.Park is buying Stratasys 3d printer. Its industrial grade top of the quality 3D printer and I hope I can learn how to use it.

We also meet his students and we had a wonderful Korean lunch with them.

2. New person that I met

I really like to read comics and I love superhero books. I wanted to go to a comic book shop but I could not find any on google maps. One day coincidently, while I was exploring the city I saw a tiny book store. There was a tiny entrance which led to a basement. At first, I was a little scared because it looked really ghetto place. But I met a really nice lady who did not speak any English. It took me 15 min to explain that I am looking for Marvel comics. in the end, I found two great comics which are really rare to find in the USA. When I was about to check out, the lady came up to me and gave me a 2-3 feet long poster of all the Marvel superhero and she also gave me a drink which had a marvel branding it. I believe she figured out that I really like Marvel.

3. What new cultural insight about Korea did you learn this week?

For some reason, all the Korean bus driver and taxi driver are older people and they just scream at drivers who block there way. Another thing I learned was how to take buses in Korea and how young people are expected to stand.

4. What did you learn about yourself?

I was really scared to communicate with people and I thought it would take a long time for me to open up but in Korea, its lot simpler, I still don't know why.

5. What is the status of your technical projects and what skills did you learn?

I am working with Mark( He is the preception person for hubo). I am helping him to setup Nvidia Jetson. Which will eventually replace hubo vision pc. With that, I am also working to get unity setup here. My goal is to import vision data into unity. Right now people who do teleoperation with hubo are not in the lab. I am meeting them after they come back form conference

Week 2: 06/10/19 - 06/24/19

1. Photo of the week

One of the interesting things, I saw was an exoskeleton which could control Jami Hubo. Unlike other exoskeletons, this one had good encoders on its joints and the movement it created was really smooth and accurate. All the members came back from the trip today and my goal is to ask one of the members to give me a quick introduction of how it works.

2. New Person that I met

Last week I met a few Indian guys from IIT. Just like me, there are a couple of students from India who came to South Korea fro VSR program. Even though I grew up in India, I have never been to college there and to be honest every one dreamt of going to IIT. But when I was talking to them it did not feel any special in fact it was no better than any colleges in India. As an undergraduate student, there are no resources for them to work at a research lab. All of the undergraduates spend their time in class and study for the end of the year exam. It made me appreciate the opportunities I am getting at UNLV and DASL.

3. What new cultural insight about Korea did you learn this week?

At first, I thought people in South Korea treats tourist nicely, but more time I spend here I realized they tear all their guest with a certain respect and honor. Where ever I go not just in restaurants people were super nice and even not all people spoke English they tried their best to help us or understand. Last week I was telling how all the bus drivers always scream and stuff but something interesting happened a couple of days ago. So I was taking the shuttle at the normal time but I didn't realize that it runs differently on Sundays. On weekends you have to take one bust from dorms to Kaist Munji campus and wait there for 30 min and take another bus from munji to the Main campus. So I was sitting at the bus station and out nowhere he invited me to sit on the bus and offered me coffee. He did not speak any English, at one point he turned up the AC and started showing me funny videos. Afterward, he took me to the main campus on scheduled time.

4. What did you learn about yourself?

One thing I noticed is that since there is no distraction here like family, friends, and classes, I am getting a lot of things done. I am staying longer at nights and I can finish up all my work on time.

5. What is the status of your technical projects and what skills did you learn?

As I said last week, I was helping mark to setup Nvidia Jetson. Which will eventually replace hubo vision pc. I learned a lot of things from that. One that Nvidia Jetson is not ideal PC to replace the whole vision PC. Nvidia Jetson runs on ARM64 architecture and all the drivers and packages are not compatible. Even though it's really good at computer vision, setting jetson up for the first time is a long process. We ended up having two computers for vison pc. One will read all the sensor data(intel nuk) and transfer everything to jetson for processing. Learning how to use jetson and communicating between two pc was important because if it works successfully, I can use it for my unity teleoperation.

Week 3: 06/24/19 - 07/1/19

1. Photo of the week

For me, the photo of the week was a Stratasys 3d printer. I was super excited about it and I could not wait to try it out. But the learning workshop for the printer was in Korean. I spent 1 hour 30 min listening to a technician and trying my best to understand something. I was able to understand enough to get started with the printer. I believe that the best way to check the quality of a printer or any machine is to test its limit. so, I had a huge part which occupied 100% of the build plate and it was 14 hours long. The results were amazing the part came out great, there was no warping and it was clean.

Things I learned

NEVER BUY STRATASYS 3D PRINTER!!!! The printer is crazy accurate but the company did some tricks in it so they can make more money. One of the biggest downsides is that the build plates are a one-time use. So after every print, you have to through away the build plate and put a new one. So people like me who do a lot of 3d printing, it is just frustrating and it's not like that they did not have any other option. They could have change small things like add a glass bed or something to improve it. You also have to buy proprietary filaments and also have to use there trashy proprietary software

2. New Person that I met

Last week, was a bit busy for me and I couldn't go out a lot but I and Jason met daniel lofaro. We attended his presentation and got a lot of good insights. We could clearly see his way of working and our lab's way of working was really similar. I also spent some time checking his wiki page.

3. What new cultural insight about Korea did you learn this week?

One of the interesting things I saw in Korea was that they recycle everything. It's part of there culture, they always separate their trash and only use plastic bags when only necessary. I know it seems like a small thing but unlike USA where people just talk about recycling, here it's everywhere. People are willing to take extra steps to have a clean environment.

4. What did you learn about yourself?

I miss my knives/multitool, back home I always carried them 24/7 and I knew I had it for small tasks like wire stripping, cutting, etc..Since I spend a lot of time in the lab working on soldering or 3d printing. At least once every couple of hours I reach to my pocket for my knife and then realize it's not there. for me, knives are one of the tools that I most relied on.

5. What is the status of your technical projects and what skills did you learn?

One of the things I learned was Nvidia Jetson was not the ideal solution for hobo's vision pc. After spending a lot of days on it, we realized that it was too much work to convert all the packages to arm64 architecture. So jetson does not have a normal processor, it has an arm64 architecture. most of the packages are not supported so you have to build it from the source. People at hub lab wanted to use jetson because it was smaller in size and had a dedicated gpu. I was able to convince them to get a tiny VR PC and put that on hubo. I was able to set it up and now it's working perfectly. Another thing I was working on last week was to brainstorm ideas about the teleoperation and talking to people at kaist about setting up port forwarding. also, I was working with intel real sense 435 depth camera.

Week 4: 07/1/19 - 07/8/19

1. Photo of the week

For me, the photo of the week was the collision detection module I am working on. Right now I and Jason are working on collision detection for rb5 using the rgbd camera. The idea is that even though rb5 has collision detection but the object has to physically touch the robot, our goal is to check collision before it happens.

2. New Person that I met

Last week, I and Jason went to a bar that Dylan recommended and the owner was super interesting. He owns a machine shop in Korea and as a part-time job, he also owns a bar. We had a great chat with him about the work he is doing and all of his projects. One of the projects was that he was trying to use computer vision to know human's mood.

3. What new cultural insight about Korea did you learn this week?

One of the interesting things I saw in Korea was that they recycle everything. It's part of there culture, they always separate their trash and only use plastic bags when only necessary. I know it seems like a small thing but unlike USA where people just talk about recycling, here it's everywhere. People are willing to take extra steps to have a clean environment.

4. What did you learn about yourself?

I still have a lot of things to learn. Spending time at KAIST is really overwhelming, I am constantly surrounded by people who are 100 times smarter than me. It's making me realize that I have to work a lot harder if I want to get to their level.

5. What is the status of your technical projects and what skills did you learn?

As described earlier I am working with collision detection for rb5. I am trying to improve on skills that I learned with projection mapping and Vr. Right now I was able to get the data from the sensor and then create an octomap. I am using that octomap to check for collision between robots urdf and surrounding.

Week 5: 07/8/19 - 07/14/19

1. Photo of the week

For me, the photo of the week was the quadruped that dr.park's student working on. I know I have already talked about it but recently they were trying to make a specialized foot for it, which will help the robot to climb on the rough metal wall. One of their methods was to sandwich an Electropermenet magnet(EPM) between the foot and a silicon pad(infused with metal dust). With some testing, they realize that if the surface is not 100% smooth the EPM won't stick directly to it, they need a soft pad in between. They wanted to transfer the magnetic field from EPM to silicon pad, then they could have a solid contact between the wall and the robot's foot. A major problem with that was that EPM has a bunch of small programmable magnets which are arranged in “alternating polarity” which creates a strongest magnetic field only 2 mm away from the surface. So if you have a silicon pad in between then the magnetic field would be almost zero. For me, this project was really fascinating I love electromagnets and I had seen this problem before. I suggested the Ph.D. student to arrange the magnets it “Halbach array” which will amplify the magnetic field on one side and decrease on the other side. We were able to do some testing and it increased the strength of the magnetic field 10 times more than the previous method. I felt even though it was not a big contribution but It felt really good.

2. New Person that I met

Last week, I went on another quest to find a comic book store in Daejon. I was not successful but I found a really nice model kit store. I bought a lot of model kits for myself and for my friends. While I was there I met Kaist Ph.D. student who was doing research on computer vision for industrial application. It was really fun to talk to him and share our experience. He also suggested some professors who teach computer vision and have free online lectures. His name was Amir and he was from Pakistan and is living in Korea for almost 3 years.

3. What new cultural insight about Korea did you learn this week?

Even though there are a lot of foreigners at KAIST. I felt Korea itself does not have a lot of different types of people. For example, the USA is pretty diverse there are different people from the dirffrent country, race, religion, etc. on the other hand, all people here are Koreans. There is no fight over minorities or the majority.

4. What did you learn about yourself?

I need to learn more programming skills. As a mechanical engineering student, I always thought that I don't need to know how to code but after seeing all the students here, I was wrong. Almost every single one here is ME student and they can code. Every time I meet a new person I always ask them one question. “What skills you wished you had before coming to hubo lab?” most of them told me that they wish they were good at programming and advance dynamics. I already signed up for cs classes for the upcoming semester.

5. What is the status of your technical projects and what skills did you learn?

Last week, I was able to do collision detection in the simulation. I could give the robot a starting and end goal and it will generate an obstacle-free path. I am working in a Ros environment called Moveit which is an industry standard for manipulators. It is a really powerful tool because it can be used to not only just control a manipulator but we can do path planning, object avoidance, teleoperation, etc..I also believe that we can use this method to do pick and place demo with hubo. Currently, I am trying to learn as much as stuff about it so I can use this for hubo teleoperation. I also talked with Dylan and we started setting up the communication. We have a plan of what we are going to do.

Week 6: 07/14/19 - 07/22/19

1. Photo of the week

For me, the photo of the week was the Korean seal that I got. last weekend I and Jason went to Seoul and over there we went to a traditional market for shopping, I found a shop who makes Korean name seal (Dojang). I love collecting antiques and traditional objects, so as soon as I saw it, I wanted to get my own Dojang. I selected a marble stamp which had a snake sculpture and I got my Korean name engraved on it. It was really interesting to see how people hand-engraved all the details.

2. New Person that I met

Last week, since we went to Seoul we meet mang, Minju, and Jisu from our lab. It was really fun to hang out with them. We went to a traditional Korean restaurant, palace and also saw the blue house. After dinner mang took us to an old Korean bar. It was really fun because it was out in open and everyone was really friendly.

3. What new cultural insight about Korea did you learn this week?

One of the things I learned about Korea is that since there is not a lot of free space here, everyone just parks their car anywhere. There were people who parked their car on the sidewalk. Another thing I found out was if someone illegally parked their car near a business, it's a business responsibility to pay the towing service. Also, taxi drivers are really rough in driving. Many people just drive through a red light without even slowing down.

4. What did you learn about yourself?

One thing I learned about myself that I some times don't like to follow conventional methods, I always want to test out different methods. For example, for our project, I am taking care of all the computer vision parts and one of my jobs was to do object localization and get XYZ of an object in relation to RB5 frame. Normally at Hubo lab, they use point cloud in order to do object localization, it is a robust method but its really computationally heavy it took them 5 seconds latency per frame. At first, I was supposed to use point cloud for my work but it just did not work out for real-time application. So I used projection mapping approach for it, I use focal length and camera calibration to get XYZ. luckily I was able to test out my non-conventional method for object localization and got good results (2.5 second per frame). I realized that here people have a specific approach to a problem that they developed over time from experiences. Some of them I agree but once in a while, I like to test out different stuff.

5. What is the status of your technical projects and what skills did you learn?

Last time when I talked about my progress, I was working on collision avoidance for RB5. But last week we took a step back and worked on simple pick and place demo. my goal was to test out my work with a real robot rather than just simulation. Also, I and Jason wanted to make sure both of part is working correctly together.

As described earlier I was doing object localization for RB5. First I used RGB feed to identify the object (actually identification was done using hubo lab's pre-trained neural network) but once we identified the object we had two approaches to find its position in the real world. Either to use Point cloud or projection mapping approach.I was able to test out both methods and we decided that we will use projection mapping approach to find XYZ. Once I had the object's position I just published it on ROS using ROS_msg. Jason's Ros wrapper was really useful because now we can make any Ros nodes and it can seamlessly be integrated with the RB5 arm.

After this demo, my goal is to make two different methods for collision avoidance. One using Move-it and another using my approach. I am doing two different approaches is because Move-it is an industrial standard for arm manipulation and I am using it as my baseline. Move-it uses point cloud for everything Also, in Back of head, I am piecing together all the skill I need in order to do pick and place using Hubo. After working on the project what I realize that if we are going to do teleoperation using Hubo we cannot just send joint state to its actuator. Hubo has limits in its joint so it won't crash to itself but there is no safeguard for objects around it. When Keitaro made his teleoperation setup it was really risky and dangerous. Now I think we can make a more reliable solution. Working with RB5 made it easier because I can make those safeguards for the arm and after I come back to dasl modify it for hubo.

Week 7: 07/22/19 - 07/29/19

1. Photo of the week

For me, the photo of the week was the resort where all the lab members and I went. It was a really beautiful place between mountains. I don't know where exactly it was but I know it was near Seoul. It was raining that day and I had a great time. We went for a yearly seminar where Hubo lab, rainbow robotics, and all the other labs talked about their current project and future plans.

2. New Person that I met

Last week I met a lot of new researchers from different partnered labs and all of us were sharing our experience while drinking and eating dinner.

3. What new cultural insight about Korea did you learn this week?

Korea has a really nice public transportation. Buses go almost anywhere and I felt Tmoney was really useful. You can use Tmoney anywhere in Korea and it makes transportation really convenient. Also Buses and subways are really clean and safe

4. What did you learn about yourself?

Last week and this week I was kind of feeling homesick. I am really enjoying my time at kaist but sometimes I get stuck on a problem and I wished I could go home and relax in my bed.

5. What is the status of your technical projects and what skills did you learn?

Last week I worked on my own object detection module. I learn basics about neural network, YOLO and TensorFlow. I am trying to create human pose recognition so I can use that information to check for collision between rb5 and the operator. I was able to use pre-trained weights to detect a person and make a bounding box. I used that bounding box to get the position of the detected person. the latency was really low and the detection was also really accurate.

Week 8: 07/29/19 - 08/5/19

5. What is the status of your technical projects and what skills did you learn?

Last week, I said I worked on my own object detection module. I learn basics about neural network, YOLO and TensorFlow. I got it working!

So what's happening in the image is there is two camera feed from two different real sense D435 sensor. I am running YOLO object detection on both of them with 15 fps per camera and with almost no latency (0.2~0.5 seconds) depending on my graphics usage. YOLO is different from other classifiers because it's running the neural net only once and it takes the whole image and use that to do predictions. This object detection module does require a POWERFULL graphics card. Currently, I am using my personal Alienware laptop with gtx 1080 and still, the temperature of my GPU rises up to 90 degrees(failing point for most GPU is 100C).

Our setup

We have one real sense camera on top(bird's eye view of rb5) and another camera is on the gripper itself.

  • We are getting XYZ of an object on the table by doing a scan from the top camera.
  • Then we are moving the RB5 on top of that object to pick it up. (RB5 is blocking the object so the top camera cannot see anything)
  • Now at this stage, if the object is not moving then Rb5 will apply picking up motion but if it's moving it will get new XYZ of the object from the second camera( one on the gripper)

Progress

we did some trial runs and we know that initial XYZ coming from the top camera(static) are correct but when we try to to get XYZ of an object from the second camera(on gripper which is a dynamic camera) it's giving us wrong values.

To do some transformation for the dynamic camera we require rb5's end-effector positions. But there is some huge latency between RB'5 computer publishing the data and us reading it.

bellow is the initial trial runs.

we are just using one camera and trying to realign the arm from updated XYZ position. In the video, we are subscribing to end-effector with the correct rate but as soon as we make the code bit complex it losing a lot of data

akshayjournal.1565092194.txt.gz · Last modified: 2019/08/06 04:49 (external edit)