Foreground rejection for parallax removal in video sequence stitching

Abstract

Parallax is a key challenge that leads to inaccurate registration and ghosting effect of objects in the result of panorama image stitching. A novel foreground rejection method is proposed in this thesis to remove parallax in video sequence stitching. Firstly, the global motion is estimated between two frames using SIFT feature matching. The foreground is obtained by applying the logical OR operators to the pixels that have high displaced frame difference. There are two groups of foreground: the near-frames and the far-frames groups. Then, voting scheme is applied in the way that only the near-frame foreground inside the area of far-frame foreground is considered as the actual foreground. The extracted foreground at this stage is mostly the edges of objects. Since the change of a foreground shape is very small, 2D translation motion from tracking algorithm is used to project foreground (edges) from other frames to the current frame. The edges from other frames are then used to refine the edge in the current frame. The 2-stage rendering is then applied to the extracted edge for the foreground area. The foreground area is refined by the foreground area of other frames mapped to the current frames by 2D translation model. The experimental results of 13 test video sequences indicated that the removal of parallax objects by the proposed foreground rejection method led to (1) more accurate registration, (2) the reduction of the ghosting effect, (3) sharper stitched result and (4) more background information in the results.