What is total variation (TV) distance?

Total variation

Total variation distance or TV distance is the measure of distance for probability distributions. It also goes by the name of statistical difference or variational distance.

TV distance simply measures the distance between two probability distributions. A formula for TV. distance is defined as follows:

$P.D$ in the formula stands for the probability distribution. Here, $s$ is the set on which we want to calculate the probability and is contained in the countable set space of $S$. So, the TV distance can be half of the sum of the probability distributions obtained on a set$s$by the difference between both distributions.

One other way of expressing the TV distance could be to take events denoted by $B$ and then say that we take the probability of these events $B$ under $P.D_1$ and $P.D_2$ one by one, after which we take their difference and then root out the maximum amongst those:

Facts surrounding TV distance

A couple of factual statements on T.V. distance have been listed below:

1. If we have two variables, $X$ and $Y$, where$X$ has a marginal distribution denoted by $D$ and $Y$ denoted by $H$ , the following statement is true:

2. If $p=\sum_x min(D_1(x), D_2(x))$then $1-p = ||D_1-D_2||$.

A simple illustration

Here, we have drawn a graph having two different PMFs Probability mass functionsof distributions and the total variation distance between them:

We can see how the total variation distance is equal to the orange and green parts between the two PMF curves.