Search⌘ K
AI Features
Home
Courses
Data Science for Non-Programmers

Duplicates

Explore how to identify and eliminate duplicate entries in datasets to improve data quality. This lesson teaches you to use Python functions like duplicated and drop_duplicates to maintain accurate and consistent data for analysis.

We'll cover the following...

Duplicates

Repeated data rows in the dataset are called duplicates. These can arise from a number of ways. The most common are:

  • The same data is entered twice by accident, such as the same article is scraped twice or booking for an online product is made twice.

  • If data is being collected in online forms or surveys and the user presses the submit button twice.

  • If data is collected from multiple ...