The CTSN consists of modules with onboard image sensors and Computer Vision processing capabilities. Relevant data is processed locally and the resulting metadata is sent over the network to a main node. In order for this to occur in an efficient manner, several different computer vision problems were addressed by the chosen hardware and associated design.
Image acquisition will be handled by the PIXYCam CMUCam. Technical specifications are as follows:
- Processor: NXP LPC4330, 204 MHz, dual core
- Image sensor: Omnivision OV9715, 1/4", 1280x800
- Lens field-of-view: 75 degrees horizontal, 47 degrees vertical
- Lens type: standard M12 (replaceable)
- Power consumption: 140mA typical
- Power input: USB input (5V) or unregulated input (6V to 10V)
- RAM: 264K bytes
- Flash: 1M bytes
- Available data outputs: UART serial, SPI, I2C, USB, digital, analog
- Dimensions: 2.1" x 2.0" x 1.4"
- Weight: 27 grams
Three specific design problems were addressed:
- Feature detection, Description and Matching
- Is that a horse, pedestrian, werewolf, ETC?
- Object Tracking
- Where is the object is travelling in the frame?
- Multi-camera Tracking
- Where in the network is the object located?
The general solution is to find existing implementations of algorithms that address each of the areas of concern. A software interface for the PIXYCam can then be utilized to plug in and test the selected algorithm.
To specifically address the first design problem, three algorithms were analyzed in great detail. These included "Speeded up Robust Features", "Binary Robust Invariant Scalable Keypoints", and "Fast Retina Keypoint". Due to availability of existing implementations in C++, and its exceptional performance in average seconds per frame, the FREAK (Fast Retina Keypoint) algorithm was chosen.
Object tracking will be handled via the "Predator" (TLD) or the "Kanade-Lucas-Tomasi" (KLT) algorithm.
Multi-camera tracking will occur by application of the "Probabilistic Occupancy Map (POM)".
Risks of CV Implementation
- Not being able to track subjects across the entire network considering the relatively small amount of the trail that is actually being observed.
- Not achieving satisfactory results with the image sensor on the PIXYCam due to variation in outdoor conditions.
CV Testing Strategy
- Gather reference video from the trail with the PIXYCam involving various subjects and under various conditions.
- Utilize this reference video to test various algorithms for both accuracy and computational performance.