All posts in “Hardware”

iPhone Application: Copenhagen Mirror

I’m now on the 5th semester of Medialogy, and we’re currently developing an iPhone application. The application lets the user use the iPhone as a camera that looks back in time, i.e. if you’re standing at Christiansborg in Copenhagen and point the iPhone at the building you should see how the area looked 500 or 1000 years ago. We also wanted this environment to be interactive, i.e. the user should be able to walk and look around. This means that we wanted to model a 3D environment of the area and synchronise the position of the user in real life to the position in this 3D environment, and also allow the user to use the iPhone as virtual reality goggles, i.e. if you tilt the iPhone upwards you see the sky, if you turn around you see what’s behind you in the 3D world, etc.

We’ve been making the models in Maya and building the application itself in Unity, since it has a pretty strong iOS support. All scripts in Unity are programmed in C#.

Basically the application will have the following features:

  • User can move around by physically moving (Assisted GPS).
  • User can look around by using the iPhone as a camera into another world (Gyroscope).
  • Weather conditions in 3D environment mimic real-life weather conditions (Network connection to DMI).
    • Position of sun is calculated based on current date and time.
    • Lighting conditions are calculated by using online information about solar radiation.
  • A graphical user interface allows the user to switch between time periods.

The application is currently running on my iPhone 4, but I have no video of it, so here’s a screenshot from Unity where one of the newer editions of Christiansborg is displayed.

We do have some screenshots from the application, though:

This is from an early build, no textures on the buildings or anything, but the gyro works, i.e. you can look around the environment by turning and moving the iPhone.

This is from the newest build. The model is the Christiansborg castle  which was there before the current one. The image is dark because the sun’s location is calculated from the real sun, and it’s gone down at this point. The arrows allow you to switch between castles.

Bad Design:
Sony’s PlayStation 3 Remote

I got my PlayStation 3 which is great and all, and I also ordered the Bluetooth remote. Already when I ordered it I winced at the horrible, horrible design choices and they’re even more apparent now that I’ve used it a little. Not only is it filled with a lot of useless buttons, it even has buttons that do the exact same thing! I recently bought a WD TV Live media player, and its remote looks like this:

wdtv-remote

It’s brilliant! It does everything it needs to with a minimum of buttons (17 to be precise). Now let’s take a look at the PlayStation 3 remote:

bd-remote-2-playstation-3

I mean, come on, this thing has 51 buttons. Who uses numbers for their DVD/Blu-ray player? I don’t even know what the “CLEAR” and “TIME” buttons do and I’ve never in my life used those colored buttons. Under those we have the four standard menu buttons, but the triangle, square, cross and circle buttons do the same thing! It’s the same with the L1-R3 buttons, which do the same as the arrow buttons underneath. It’s ridiculous, so in the WD TV Live spirit I decided to photoshop it into this:

bd-remote-2-playstation-3_new

Honestly, do you need more?

Other than the design flaws the remote seems to work fine, it’s weird not having to point it anywhere though.

Jailbreaking my iPhone

I’ve recently started considering jailbreaking my iPhone because I’ve started getting these spam calls from Somalia – sometimes 15 calls per night – and I know that there’s a jailbreak app which makes it possible to block calls. So, after stumbling upon an extremely simple jailbreaking guide, I went ahead and followed it, thinking I could always just restore my iPhone if something got messed up.

First of, the jailbreaking process itself is incredibly easy, it probably took less than 60 seconds. Then I started exploring the new world of Cydia, an application which makes it possible to download other applications for jailbroken iPhones and iPod Touches. It looks very similar to the official App Store, i.e. you find an app and press download. After surfing various sites for neat apps, I now use the following:

Winterboard

This app manages the themes you are using, I settled on Matte Nano and Glasklart, as seen in this screenshot:

photo 2

Backgrounder: This simply lets applications run in the background.

Cycorder: A video-recording app. I haven’t tested the quality yet.

CyDelete: Makes it possible to delete Cydia apps without opening the Cydia app itself.

Five Icon Dock: Makes it possible to have 5 apps in the dock.

iBlacklist: This app blocks incoming text messages and phone calls, bye, bye, African spammers.

LockInfo: Customizes the lock screen, as seen in this screenshot:

photo
I still need to find a better wallpaper, but this app is amazing.

OpenSSH: Makes it possible to use SSH to transfer files (themes and icons for example) from your computer to the iPhone.

SBSettings: Very useful app which is accessible from anywhere on the phone. You just slide your finger on the status bar and a GUI drops down making it possible to enable/disable 3G, WiFi, Bluetooth, etc. on the fly.

And that’s it for now. At the moment I’m kind of confused why I didn’t do this a year ago. Oh, well.

Image Processing and Motion Tracking

One month into the 3rd semester, and I finally feel like I’m learning something useful. We’ve had a lot of programming classes, focusing on C++ and OpenCV, a computer vision library focusing on real-time image processing. The point of all this is to learn how to make tracking systems (think the big screen from Minority Report) and today we finally got a first, practical look at how to apply all of this.

jonas_motion

Jonas in the hot motion tracking suit.

Most people have probably seen this in behind the scenes videos from movies and computer games (this is a simple version of how they made Gollum). The procedure is simple: you adjust the aperture, shutter speed, lighting settings, etc. until all you see are the suit’s tracking points in the dark. Using an infrared camera makes it easier, but this was only a test. But then you have to track the points, and this is where image processing comes in.

Screen shot 2009-10-05 at Mon, Oct 5, 2009

Excerpt from the source code (turning a grayscale image into a binary image).

Now you have to isolate and track the interesting points (called blobs), in this case the reflective buttons, by applying some image processing techniques. First, you turn the color image/video into grayscale with a built-in function in OpenCV, and then you convert it to binary, which means that we only get two colors in the image: black (0) and white (255). To convert the video images we make all pixel values above or equal to some threshold (235, in this case) white and all below black.

The original video.

The video converted to binary.

The reason for doing this is, as mentioned, that we want to be able to track something, so we have to remove noise, i.e. stuff we don’t want to track. In this case we want everything but the reflective buttons to be black, so we apply the steps above. As you can see in the last video, there’s still a lot of noise (light at the edge of the screen, Jonas’ face and hands when he gets close, etc.), but in the middle of the video we have a sequence with a “clean” region, where most noise around Jonas’ reflective buttons are gone. Then we can start tracking. However, we will remove more noise by applying various techniques (mean filter, dilation, etc.) first, but that’s for another post.

Windows Search, you fail

How do you fuck this up?

windows_search_fail1