Color Target Tracking and Acquisition Using CMUcam

To make a robot follow a colored target using the CMUcam2, or a CMUcam3 with CMUcam2 firmware, you must have the following

-          Robot chase

-          CMUcam2 or CMUcam3

-          Microcontroller or Computer with serial port (TTL or RS232)

The first step is to connect your microcontroller or computer to your robot chase so the robot chase can be controlled by the microcontroller or computer.† From this point on I will assume that you are using a computer to control the robot.

Now that the robot can be controlled by the computer you can connect the CMUcam to the computer via the serial port.† This tutorial assumes that you are using the CMUcam2 or the CMUcam3 with the CMUcam2 firmware.† For serial port setup, pin out configuration and power please see the links above for datasheets for the given CMUcam.

The CMUcam2 has a useful feature where it will automatically track the average color in the center of the screen when the TW command is sent.† TW stands for Track Window.†† In order to do this please follow the following steps:

-          Connect to the CMUcam to your computer via serial, use the datasheets listed above for the pin outs and connection details

-          Connect power to the CMUcam, see datasheets above.

-          Power on the CMUcam

-          Place the mono colored object about 6 inches in front of the CMUcamís lends.

-          After a minimum of 5 seconds after power on send the following ASCII characters to the CMUcam: TW\r (where \r is a carriage return).† This means Track Window.† It will take the average color of the window and track that color.

-          The CMUcam will return the ASCII: ACK telling you that it received your command.

-          The CMUcam will then continue to return T packets until told to stop:

o   Type T packet: (T, mx, my, x1, y1, x2, y2, pixels, confidence, \r)

o   This is the return packet from a color tracking or frame differencing command.

o   mx - The middle of mass x value

o   my - The middle of mass y value

o   x1 - The left most cornerís x value

o   y1 - The left most cornerís y value

o   x2 - The right most cornerís x value

o   y2 -The right most cornerís y value

o   pixels ĖNumber of Pixels in the tracked region, scaled and capped at 255: (pixels+4)/8

o   confidence -The (# of pixels / area)*256 of the bounded rectangle and capped at 255

-          Now use mx, my, and the confidence to track the colored object on the screen.


The example below is a robot that is programmed using a simple P controller for tracking the target when it is found by the CMU cam with a high enough confidence.† If the confidence is not high enough then the robot will simple rotate in a circle looking for the target.† See the video above for the robot in action.

Above is a picture of the parts of the robot and the playing field.† The picture shows the location of the CMUcam, the front of the robot and the targetís location.