Search⌘ K
AI Features
Home
Courses
An Introduction to Entity Resolution in Python

Similarity of Numeric Attributes

Explore how to engineer similarity features for numeric data such as amounts, dates, and geocodes in entity resolution. Learn to use RecordLinkage's built-in functions and create custom similarity measures, improving your ability to identify duplicate records efficiently in Python.

We'll cover the following...

Duplicate payments likely have similar amounts and transaction dates. Duplicate locations have similar geocodes. All three are numeric (vector) attributes. Let’s use a tiny dataset to show how to configure similarity features for numeric attributes using the RecordLinkage API.

C++
import pandas as pd
df = pd.DataFrame({'amount': [123.45, 110.66, 120.55],
                   'date': pd.to_datetime(['2022-06-25', '2022-04-01', '2022-07-03']),
                   'lon': [6.54321, 6.98765, 6.54398],
                   'lat': [51.23456, 51.22222, 51.23111]})
print(df)

Note that dates can be interpreted as strings or numeric values in units of, for example, days. Our focus here is on the numeric interpretation.

Custom comparer

A similarity feature is limited to 11 for any given record pair by comparing a selected attribute. How similar are the amounts ...