Posts tagged copenhagen institute of technology

One month into the 3rd semester, and I finally feel like I’m learning something useful. We’ve had a lot of programming classes, focusing on C++ and OpenCV, a computer vision library focusing on real-time image processing. The point of all this is to learn how to make tracking systems (think the big screen from Minority Report) and today we finally got a first, practical look at how to apply all of this.

Jonas in the hot motion tracking suit.

Most people have probably seen this in behind the scenes videos from movies and computer games (this is a simple version of how they made Gollum). The procedure is simple: you adjust the aperture, shutter speed, lighting settings, etc. until all you see are the suit’s tracking points in the dark. Using an infrared camera makes it easier, but this was only a test. But then you have to track the points, and this is where image processing comes in.

Excerpt from the source code (turning a grayscale image into a binary image).

Now you have to isolate and track the interesting points (called blobs), in this case the reflective buttons, by applying some image processing techniques. First, you turn the color image/video into grayscale with a built-in function in OpenCV, and then you convert it to binary, which means that we only get two colors in the image: black (0) and white (255). To convert the video images we make all pixel values above or equal to some threshold (235, in this case) white and all below black.

The original video.

The video converted to binary.

The reason for doing this is, as mentioned, that we want to be able to track something, so we have to remove noise, i.e. stuff we don’t want to track. In this case we want everything but the reflective buttons to be black, so we apply the steps above. As you can see in the last video, there’s still a lot of noise (light at the edge of the screen, Jonas’ face and hands when he gets close, etc.), but in the middle of the video we have a sequence with a “clean” region, where most noise around Jonas’ reflective buttons are gone. Then we can start tracking. However, we will remove more noise by applying various techniques (mean filter, dilation, etc.) first, but that’s for another post.