Limits of motion-background segmentation using fundamental matrix estimation

Images of a moving object acquired by a static camera typically contain two groups of image correspondences belonging to the object in motion and the static (or nearly static) background. For objects with relatively small motion, there is little distinction between the effect of motion and the background noise. As the type of motion is also not known in advance, the fundamental matrix motion model is usually applied in most vision algorithms. In this paper, we study the separability of a small motion from the static background via fundamental matrix estimation and introduce the necessary conditions for successful motion-background separation. We show that a pure translational motion in the above framework is inseparable from the static background regardless of its magnitude. An extensive set of controlled experiments have been conducted to validate the findings and to quantify the necessary condition (in terms of the rotational angle) for successful separation of a general motion from a static background.