The CTSN consists of modules with onboard image sensors and Computer Vision processing capabilities. Relevant data is processed locally and the resulting metadata is sent over the network to a main node. In order for this to occur in an efficient manner, several different computer vision problems were addressed by the chosen hardware and associated design.


Image acquisition will be handled by the PIXYCam CMUCam. Technical specifications are as follows:

Three specific design problems were addressed:

  1. Feature detection, Description and Matching
    • Is that a horse, pedestrian, werewolf, ETC?
  2. Object Tracking
    • Where is the object is travelling in the frame?
  3. Multi-camera Tracking
    • Where in the network is the object located?

The general solution is to find existing implementations of algorithms that address each of the areas of concern. A software interface for the PIXYCam can then be utilized to plug in and test the selected algorithm.

To specifically address the first design problem, three algorithms were analyzed in great detail. These included "Speeded up Robust Features", "Binary Robust Invariant Scalable Keypoints", and "Fast Retina Keypoint". Due to availability of existing implementations in C++, and its exceptional performance in average seconds per frame, the FREAK (Fast Retina Keypoint) algorithm was chosen.

Object tracking will be handled via the "Predator" (TLD) or the "Kanade-Lucas-Tomasi" (KLT) algorithm.

Multi-camera tracking will occur by application of the "Probabilistic Occupancy Map (POM)".

Risks of CV Implementation

CV Testing Strategy