This device is not compatible.


Exploring Public Sentiment on Twitter with NLP

In this project, we will learn how to apply natural language processing (NLP) using Python libraries VADER and Gensim, to understand people’s views as expressed in Twitter tweets.

Exploring Public Sentiment on Twitter with NLP

You will learn to:

Handle unstructured data using Python.

Conduct sentiment analysis with natural language processing (NLP).

Identify and visualize sentiment trends over time.

Apply topic modeling with Gensim.


Natural Language Processing

Sentiment Analysis

Topic Modeling

Data Visualization


Basic knowledge of Python programming

Familiarity with pandas for data handling

Basic understanding of NLP concepts and techniques

Basic understanding of text processing and sentiment analysis




Gensim logo



Project Description

Sentiment analysis is essential to learn how businesses and researchers grasp public opinion and allows real-time tracking of general sentiment toward products, services, events, or trending topics. Especially in platforms like X (formerly Twitter), with its wealth of user opinions, sentiment analysis is essential in understanding the social media landscape.

In this project, we will learn how to analyze sentiments on Twitter by focusing on a dataset containing more than 100,000 opinions about COVID-19 from users worldwide.

The challenge lies in managing and interpreting a diverse collection of text data. With Python, and libraries like pandas and NLTK, we will learn to process the raw, unstructured Twitter data and turn it into an organized format suitable for analysis. Next, we will familiarize ourselves with the VADER sentiment analysis designed for the intricacies of social media language.

By the end of this project, we will be able to analyze global sentiments about COVID-19 and skillfully visualize COVID-19 data. Whether new to NLP or looking to refine our data handling skills, this project offers a solid foundation and prepares us for more advanced NLP challenges.

Project Tasks


Getting Started

Task 0: Introduction

Task 1: Import the Libraries


Data Collection and Preprocessing

Task 2: Load the Dataset and Have a First Look

Task 3: Basic Text Preprocessing

Task 4: Implement Advanced Text Preprocessing


Sentiment Analysis

Task 5: Perform Sentiment Analysis with Vader Library

Task 6: Classify the Tweets into Positive, Neutral, and Negative


Trend Analysis and Visualization

Task 7: Display the Evolution of Sentiment Over Time

Task 8: Use Word Clouds to Visualize the Words Used in Sentiments

Task 9: Display the Sentiment on a Geographical Heatmap


Topic Modeling

Task 10: Train the LDA (Gensim) Model

Task 11: Evaluate the Model

Task 12: Classify Twitter Tweets into Topics


Interpretation of Results

Task 13: Identify Relationships between the Sentiment and the Topic

Task 14: Interpret the Topic Modeling Results

Task 15: Compile the Findings into a Final Report with the NBConvert Package