A real-time 3D tracking system using multiple cameras

Abstract

Tracking an object in three dimensional space is a major issue in computer vision which is normally solved through the extraction of representative features of the object, and two-dimension coordinates of the series of these image features are used to compute the position of the object. Typical system uses a binocular stereovision system. For environment with obstruction, only two cameras is not practical, multiple cameras are used instead. When multiple cameras are used, a certain similarity measure among extracted features from any two stereoscopic images helps to match the correspondences. In this way, three-dimensional measurement can be obtained from the 2-D coordinate of the features extracted from the different cameras. In this research, a multiple cameras system (four cameras) and PC-cluster (Two microcomputers) are used for estimating both position and velocity of a specified moving object. Noise filtering and features extraction of images are performed in the PC-cluster at video rate. Then, the extracted features from every camera will be used to locate the object. This is done in the main computer. The synchronization mechanism between computers has been developed using PCI-to-PCI data movers with fiber optic connection. The developed system can use both Tsai's method and Zhang's method for calibrating the system. For Zhang's method, we purpose a modified distortion model to reduce the computation time in 3-D reconstruction process. In our experiments, we setup the system to track 3-D paths which are generated by the PA10 robotic arm. The results show that the system can track both position and velocity of moving object in real-time with acceptable accuracy. Moreover, we show that the system can be adapted to be used for the reverse engineering application.