BERTSUM for Extractive Summary

Learn different ways to use BERTSUM to find the probability of a sentence being important enough to include in an extractive summary.

In extractive summarization, we create a summary by selecting only the important sentences from the given text. To perform extractive summarization, we obtain the representation of every sentence in the given text using a pre-trained BERT model.

Now let's see how to use BERTSUM in the following three ways:

  • BERTSUM with a simple classifier

  • BERTSUM with an inter-sentence transformer


BERTSUM with a classifier

We feed the representation of a sentence to a simple binary classifier, and the classifier tells us whether the sentence is important or not. That is, the classifier returns the probability of the sentence being included in the summary. The classification layer is often called the summarization layer. This is shown in the following figure:

Get hands-on with 1200+ tech skills courses.