HomeCoursesAn Introduction to Entity Resolution in Python

Advanced

Updated 3 months ago

An Introduction to Entity Resolution in Python
Save

Explore entity resolution in Python, including use cases, semantic preprocessing, graph clustering, and weak supervision. Boost business value with hands-on coding and strategic decisions.

Join 2.7 million developers at

Overview

Content

Reviews

A typical business stores data across multiple systems, including ERPs for operations, a CRM for marketing, files, notebooks, and BI apps for other purposes. Records of the same customer (entity) exist in multiple places, likely not in sync across nor unique within sources. This inconsistent situation generates an opportunity for us to drive business value by cross-referencing and deduplicating records with entity resolution. This course covers business acumen and hands-on coding. It starts with several business cases and a quick introduction to entity resolution in Python. Then, it explores semantic-preserving preprocessing, similarity feature engineering, graph clustering, weak supervision, confident learning, and integration. As a developer, you’ll increase your company’s business value by developing and deploying entity resolution pipelines. As a decision-maker, you’ll know which solution best suits your business cases and how to negotiate the best value for your money.

A typical business stores data across multiple systems, including ERPs for operations, a CRM for marketing, files, notebooks, an...Show More

WHAT YOU'LL LEARN

The ability to deduplicate records using Python

Familiarity with an entity resolution framework and business cases

An understanding of semantic similarity and search

Experience with classification in the context of entity resolution

Hands-on experience in data-centric AI using weak supervision and confident learning

The ability to deduplicate records using Python

Content

63 Lessons1 Project7 Quizzes1 Assessment

Introduction to Entity Resolution and Applications

7 Lessons

Learn how to use entity resolution techniques in Python to improve data quality and integration.

About the Course

Deduplication vs. Linkage

Location Resolution: Geocoding

Master Data Management

Analytics in a Fragmented Data Landscape

Direct Impact with Payment Resolution

Real-Time Entity Resolution

A Quickstart Guide Using the RecordLinkage Package

8 Lessons

Get started with entity resolution in Python using text preprocessing, similarity scoring, and evaluation techniques.

Solver’s Kitchen—Installation and Service for Restaurants

Effective Text Preprocessing

Similarity Features

Match vs. No-Match

Evaluate the Match Quality

Geocoders for Location Resolution

Sell with Insights

Quiz on Entity Resolution Fundamentals

Preprocessing

7 Lessons

Work your way through preprocessing text and location data for enhanced entity resolution.

Why Preprocessing Is Important

Preprocessing Using pandas

Removing and Replacing Tokens

Phonetic Algorithms

Preprocess Locations Using Open Data

Categories from Unstructured Texts

Quiz on Preprocessing

Indexing

6 Lessons

Apply your skills to enhance the efficiency of entity resolution using various indexing techniques.

Significance of Indexing

Equality-Based Indexing

Sort-Based Indexing

Combine Indexing Techniques

K-Nearest Embeddings Blocking

Quiz on Indexing

Feature Engineering

5 Lessons

Deepen your knowledge of feature engineering for entity resolution, exploring various similarity methods.

Introduce Similarity

Similarity of Numeric Attributes

Edit- and Substring-Based Similarity

Embedding-Based Similarity

Quiz on Similarity Features

Pairwise Matching

12 Lessons

See how it works to tackle class imbalances and enhance binary classification in entity resolution.

Binary Classification in Entity Resolution

Learning from Labeled Examples

Targeting Uncertainty with Active Learning

Fighting Label Errors

Programmatic Labeling

Undersampling with NearMiss

Oversampling with Text Augmentation

Interpretable Matching

Deep Matching

Exercise: Undersampling

Solution: Undersampling

Quiz on Pairwise Matching

Clustering

6 Lessons

Piece together the parts of clustering techniques to improve classification accuracy in entity resolution.

Motivating Clustering

Transitive Clustering

Merge Center Clustering (MCC)

Markov Clustering

Manual Review and Labeling

Quiz on Clustering

Integration

8 Lessons

Step through integrating entity resolution in data stacks via various platforms and services.

Motivating Integration

Local Stack with DuckDB and dbt

In the Cloud with Snowflake and dbt

SQL-First Entity Resolution with Splink

Entity Resolution as a Service

Exercise: Indexing with Splink

Solution: Indexing with Splink

Quiz on Integration

Entity Resolution Fundamentals

Assessment

Matching Products Across Two Online Shops

Project

Conclusion

1 Lessons

Look at essential insights and skills for effective entity resolution in various systems.

Wrap Up

10.

Appendix

3 Lessons

Examine batch geocoding, vector search with LanceDB, and essential resources for entity resolution.

Batch Geocode with Geoapify and Python

Embeddings and Vector Search with LanceDB

Glossary

Auto-Tagging System for Content Categorization

Project

Certificate of Completion

Showcase your accomplishment by sharing your certificate of completion.

Course Author:

Paul Kinsvater

Developed by MAANG Engineers

Every Educative resource is designed by our team of ex-MAANG software engineers and PhD computer science educators — subject matter experts who’ve shipped production code at scale and taught the theory behind it. The goal is to get you hands-on with the skills you need to stay ahead in today's constantly evolving tech landscape. No videos, no fluff — just interactive, project-based learning with personalized feedback that adapts to your goals and experience.

Trusted by 2.7 million developers working at companies

"These are high-quality courses. Trust me. I own around 10 and the price is worth it for the content quality. EducativeInc came at the right time in my career. I'm understanding topics better than with any book or online video tutorial I've done. Truly made for developers. Thanks"

Anthony Walker

@_webarchitect_

"Just finished my first full #ML course: Machine learning for Software Engineers from Educative, Inc. ... Highly recommend!"

Evan Dunbar

ML Engineer

"You guys are the gold standard of crash-courses... Narrow enough that it doesn't need years of study or a full blown book to get the gist, but broad enough that an afternoon of Googling doesn't cut it."

Software Developer

Carlos Matias La Borde

"I spend my days and nights on Educative. It is indispensable. It is such a unique and reader-friendly site"

Souvik Kundu

Front-end Developer

"Your courses are simply awesome, the depth they go into and the breadth of coverage is so good that I don't have to refer to 10 different websites looking for interview topics and content."

Vinay Krishnaiah

Software Developer

Hands-on Learning Powered by AI

See how Educative uses AI to make your learning more immersive than ever before.

Instant Code Feedback

Evaluate and debug your code with the click of a button. Get real-time feedback on test cases, including time and space complexity of your solutions.

AI Mock Interviews

Test your skills in a simulated interview setting. Receive personalized feedback based on your performance. Available for Coding Interviews, System Design, and more.

Adaptive Learning

At various checkpoints throughout Educative courses, you will be prompted to take a quick assessment. Receive a condensed curriculum tailored to your strengths and skill gaps.

Explain with AI

Select any text within any Educative course, and get an instant explanation — without ever leaving your browser.

AI Code Mentor

AI Code Mentor helps you quickly identify errors in your code, learn from your mistakes, and nudge you in the right direction — just like a 1:1 tutor!

Course

Machine Learning System Design

Gain insights into ML system design, state-of-the-art techniques, and best practices for scalable production. Learn from top researchers and stand out in your next ML interview.

1 h 30 m

intermediate

Course

Applied Machine Learning: Deep Learning for Industry

Gain insights into industry-level machine learning by applying advanced TensorFlow and Python techniques. Explore efficient methods for creating scalable models, from data pipelines to deployment and inference.

3 h

advanced

Course

Image Recognition with Machine Learning

Gain insights into image data processing and CNNs for image classification using TensorFlow. Delve into CNN architectures and their applications, requiring Python and TensorFlow knowledge.

7 h

advanced

Course

Python 3: From Beginner to Advanced

Gain insights into Python by exploring basic to advanced concepts, including loops, classes, and GUIs with Tkinter. Delve into testing and become a productive Python programmer.

4 h

beginner

Course

Fundamentals of Machine Learning: A Pythonic Introduction

Learn machine learning with scikit-learn, covering supervised learning, clustering, regression, SVMs, autoencoders, and ensemble methods through practical Python projects.

14 h

beginner

Course

Natural Language Processing with Machine Learning

Gain insights into processing text data, creating word embeddings, and using LSTMs for semantic analysis and machine translation. Explore industry-relevant NLP techniques with Python and TensorFlow.

9 h

advanced

Course

Machine Learning System Design

intermediate

1 hour 30 min

Course

Applied Machine Learning: Deep Learning for Industry

advanced

3 hour

Course

Image Recognition with Machine Learning