Bag-of-Words, TF-IDF, and Cosine Similarity

Explore techniques to convert product reviews into numerical features using bag-of-words and TF-IDF. Learn how to weight distinct terms and rank similarities between documents with cosine similarity to improve text retrieval and analysis. This lesson helps you understand how to represent text data numerically and measure its relevance across a corpus.

We'll cover the following...

Count terms with bag-of-words
Weight distinctive terms with TF-IDF
Rank reviews with cosine similarity
Check the retrieval rules

A complaint reading battery drains after two hours needs a way to find related past reviews without hand-written matching rules. Because normalization has already turned text into consistent tokens, we can now convert each review into numbers, weight the useful terms, and rank reviews by similarity.

A corpus is the complete set of documents we search. For this lesson, the corpus contains short product reviews such as battery drains quickly, battery lasts all day, and charger stopped working. The vocabulary is the set of unique tokens found across that corpus. A bag-of-words (BoW) represents each document by counting how often every vocabulary term appears.

BoW ignores word order. The reviews battery drains quickly and quickly drains battery therefore receive identical counts. That loss is deliberate here because the immediate task is retrieval based on shared terms, not an interpretation of sentence structure.

Count terms with bag-of-words

The retrieval pipeline begins by fitting CountVectorizer on the review corpus. CountVectorizer creates a vocabulary and produces a document-term matrix, where each row represents one review and each column represents one vocabulary term.

Python

from sklearn.feature_extraction.text import CountVectorizer
def main() -> None:
    reviews = [
        "battery drains quickly",
        "battery lasts all day",
        "charger stopped working",
        "battery drains after two hours",
        "not charging after one week",
        "fast shipping and great packaging",
        "camera quality is amazing",
        "battery not charging",
        "screen cracked on arrival",
        "charges slowly but battery lasts",
        "customer support was helpful",
        "battery drains quickly after charging overnight",
    ]
    vectorizer = CountVectorizer()
    dtm = vectorizer.fit_transform(reviews)
    print(f"Document-term matrix shape: {dtm.shape}")
    print("Vocabulary (token -> column index):")
    print(vectorizer.vocabulary_)
    feature_names = vectorizer.get_feature_names_out()
    print("\nDense document-term matrix (first 3 reviews):")
    print(dtm[:3].toarray())
    print("Feature names (column order):")
    print(feature_names)
    row0 = dtm[0]
    row0_coo = row0.tocoo()
    print("\nNonzero token counts for review 0:")
    print(f"Review 0 text: {reviews[0]}")
    for col_idx, count in zip(row0_coo.col, row0_coo.data):
        print(f"  {feature_names[col_idx]}: {int(count)}")
if __name__ == "__main__":
    main()

1.Get Started

2.SQL for Data Analysis

3.From SQL to Python: Working with Data in NumPy and pandas

4.Statistics for Data Science

5.Data Cleaning and Exploratory Analysis

6.Feature Engineering

7.Data Visualization and Storytelling

8.Framing a Responsible ML Problem

9.Linear Regression and Model Evaluation

10.Classification Models

11.Generalization, Validation, and Regularization

12.Decision Trees

13.Classification Evaluation

14.Bagging and Random Forests

15.Gradient Boosting

16.Model Interpretability and Deployment

17.Unsupervised Learning: Clustering and Dimensionality Reduction

18.Time Series Fundamentals

19.Forecasting Models and Evaluation

20.Text Analytics

21.A/B Testing and Experimentation

22.Recommender Systems

23.Wrapping Up

Bag-of-Words, TF-IDF, and Cosine Similarity

Count terms with bag-of-words