What is background subtraction?

Overview

In background subtraction, we subtract the unnecessary background from the image and take only the vital information of the foreground object. The input for background subtraction is usually a video stream or a series of images.

The goal of background subtraction is to separate the background from the object and generate a foreground mask. This mask is of the exact dimensions as the original image, with ones in the place of the foreground object and zeros in the background.

The background in the original image or a video stream frame is the background model.

As shown in the figure above, we generate a foreground mask that extracts just the object under consideration when applied to the original image. There're several methods to perform background subtraction. Each of them has its pros and cons. The most common and simple one is frame difference.

Frame difference

In the frame difference method, we take a previous frame from our input and an absolute difference from the current frame. Here, we assume that the previous frame is our background model (static background), and the current frame has the object we want to focus on. In this way, the static background will be zero.

It would be an ideal case if the background stayed static. But with the object's movement in the frame, we may have a difference between light and shadows. Therefore, we apply thresholding In thresholding, we take a threshold and depending on whether an input value is greater than the threshold, we include it and vice versato get the foreground mask. We'll change this threshold to get better results. We can choose a threshold value manually or use an automatic thresholding algorithm.

Its mathematical representation is:

Advantages

It's helpful to use the frame difference method for the below reasons:

• It is the simplest approach to implement.
• It demands less computational cost, so it's feasible in most cases where accuracy can be compromised to some extent.

Drawbacks

Frame difference is a fairly simple method, however, there're drawbacks that are as follows:

• There should be a clear difference between the background and the foreground. If the object is not clearly visible, the foreground mask may not include important visuals of the object.
• Sometimes, we may not be able to get a static background image. In this case, we would have to construct a fake background.
• Video streams are often noisy. This can result in a noisy foreground mask.

Applications

We can use the background subtraction in:

• Object tracking
• Human-Computer interaction
• Traffic monitoring
• Video surveillance
• Content-based video encoding