Automatic Image De-fencing System

Tourists and Wild-life photographers are often hindered in capturing their cherished images/videos by a fence that limits accessibility to the scene of interest. The situation has been exacerbated by growing concerns of security at public places and a need exists to provide a tool that can be used for post-processing such “fenced videos to produce a “de-fenced” image. There are several challenges in this problem, we identify them as 1. Robust detection of fence/occlusions. 2. Estimating pixel motion of background scenes. 3. Filling in the fence/occlusions by utilizing information in multiple frames of the input video. In this work, we aim to build an automatic post-processing tool that can efficiently rid the input video of occlusion artifacts like fences. Our work is distinguished by two major contributions. The first is the introduction of learning based technique to detect the fences patterns with complicated backgrounds. The second is the formulation of objective function and further minimization through loopy belief propagation to fill-in the fence pixels from multiple frames. We observe that grids of Histogram of oriented gradients descriptor using Support vector machines based classifier and Convolutional based deep neural networks significantly outperforms in terms of detection accuracy of texels. We present results of experiments using several real-world videos to demonstrate the effectiveness of the proposed fence detection and de-fencing algorithm.