Search⌘ K
AI Features
Home
Courses
Performing Natural Language Processing with R

Creating a List of Frequent Terms

Explore how to create lists of frequent terms from text documents using R's tm package. Understand the role of corpora, document-term matrices, and frequency thresholds to extract meaningful insights from text data for NLP tasks.

We'll cover the following...

Frequent terms

Frequent terms play an important role in natural language processing because they can provide insights into the underlying patterns, structures, key themes, and topics present in a document. They can be used for information retrieval tasks such as search engines, recommender systems, and question-answering systems.

Here’s a code example to find frequent terms in a document.

R
library(tm, quietly = TRUE)
# find terms which appear 400 or more times in the document
Corpus(DirSource(pattern = "mws_.+txt",
                  directory = "data")) |>
  DocumentTermMatrix( ) |>
  findFreqTerms(lowfreq = 400)
...