Python Webcam IR Tracking

Python WebCam IR Tracking

This is a tutorial on how to turn your webcam into an infrared tracking device using Python and a simple webcam hack. We will map the IR data to the mouse. This is much like connecting a Wiimote to your computer (via bluetooth) and using PyGlove to emulate the mouse. Except, we’re using python and cutting the $40 Wiimote out of the equation.

First the webcam. We need to filter out all light except infrared. To do this simple place film negative over the lens of the cam. Yes, film; the stuff we used to use before everything turned digital. You may have some old negatives from developed pictures. You don’t need the part if the film that has the images on it, just the black ends. Going to a local camera shop, could find useful. Most shops throw away the exposed negatives away. I layered mine with three layers of film, and taped it to my laptops webcam. To test it and see if it worked, try to view your webcam. If the video feed is black, point a tv remote (or any remote lying around) you should see a white dot appear in the video. You can also use open flame, like a candle or a lighter to test it. After you got that working, its time for the good stuff.

I am using Python 2.6, I’m not sure if it will work on Python 3.x or not. You can download Python at http://www.python.org/ If you’re not familiar with the language, than I suggest you look through the docs.

We also need some third-party library’s to make this work.

First we import our library’s that we need:

#IRClass.py
from VideoCapture import Device
import Image
from ctypes import *

Next we create a new class and initialize the handle for the webcam.


class IR:
    def __init__(self):
        self.cam = Device()
        self.user = windll.user32 #Handle for the mouse
        rp = self.cam.getImage()
        self.res = (rp.size[0],rp.size[1])

We create a new method to search an image for a IR dot. This is done by using getprojection(). getprojection() returns two lists containing boolean pixel data. If the pixel is black or 0,0,0 it’s a 0. If the pixel > 0,0,0 it’s a 1. That is why we needed the film over the lens so it will be black until an IR and only IR light is present.

def irSingle(self,im):
    imxy = im.getprojection()
    imx = imxy[0] #x coordinates
    imy = imxy[1] #y coordinates
    x = imx.index(1) #try to find a 1
    y = imy.index(1) #which is really our IR dot
    return (x,y)

Now we use the handle to grab an image instance for processing. First we need to resize the image to the resolution of the computer screen to get full range of the mouse. I am using a widescreen laptop, so my resolution is 1440×900.  We put the code inside a try/except statement because irSingle may return None.

def xySinglePos(self, scale=(1440,900)):
    try:
       im = self.cam.getImage()
       im.resize(scale)
       x,y = IR.irSingle(self,im)
       return (x,y)
    except ValueError:
       pass

Finally we set the mouse position with the x,y values we got from our other methods. Note: We are setting the virtual cursor, not the system cursor. This will not work with most video games you buy at the store. Online flash games and a lot of other things it will work with. Most games like Halo and Quake use the system mouse, which is a step below the virtual, making the system mouse the dominate one.

def setPos(self,x,y):
    self.user.SetCursorPos(x,y)

To use this new class:

#IRtest.py
import IRClass

ir = IRClass.IR()
while 1:
    x,y = ir.xySinglePos(scale=(1440,900)
    ir.user.SetCursorPos(x,y)

I hope this was useful to someone. It took me a long time to find everything I needed. And it was a great learning experience. I will be posting on how to incorporate PyGame, so we can see what the camera see’s. Maybe some games using this. I have also used it to “mock” the portal view/frustum correction Johnny Lee did with the Wiimote (http://www.youtube.com/watch?v=Jd3-eiid-Uw). I use Blender3D for the models and Panda3D for the actual game play.

This whole project was inspired by Mr. Lee and what he did with the Wiimote.

Advertisements

About Tech B.

I am currently employed as a tech support rep for Frontier DSL internet and Dail-Up service. If you live on the East Coast and have Frontier or the old Verizon, you may have spoken with me at one point. I do side jobs programming and building things for people. I know Python better than any language in my toolbox. I can also develop Android Apps, which are Java based. Other languages include C , VB/VB.NET, some C#, PBASIC, Batch, Javascript, and some PHP. I love microcontrollers and interfacing with the outside world. I am currently working with Arduinos, which are amazing. Also I can work with the Basic Stamp family of microcontrollers; future development includes FPGA and embedded Linux. I was going to school, but have financial issues at the moment that are keeping me from the books. I plan on going back in the next couple semesters and finishing up my Associates in IT, then perusing a BS in computer science.
This entry was posted in Programming Python and tagged , , , , , , , , , , . Bookmark the permalink.

6 Responses to Python Webcam IR Tracking

  1. Kris English says:

    Hi
    My name is Kris English, and I have a question regarding IR tracking.
    Is it posible to get a camera to track an IR beacon, and still film real world proceedings?

    Thanks Kris

  2. Pritesh Desai says:

    C:\OpenCV2.1\samples\python>IRTest.py
    Traceback (most recent call last):
    File “C:\OpenCV2.1\samples\python\IRTest.py”, line 6, in
    x,y = ir.xySinglePos(scale=(1920,1080))
    AttributeError: IR instance has no attribute ‘xySinglePos’

    I get this error. what should I do?

  3. Eric says:

    Greetings, have you ported this to linux yet?

  4. Sally says:

    My imxy seems to be resulting with a majority of 1s, as if it is detecting a lot more light than it should. I tried turning off all lights as well as the laptop display and also clamped on numerous film to the camera and I still get giant arrays of 1s for imx and imy. Any suggestions?
    I am using a remote as the ir until I set up a lone-standing led.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s