Encoding Categorical Variables
Explore how to transform categorical variables into numeric features for machine learning models. Understand the purpose and application of one-hot, label, and target encoding. Learn to select the right encoding method based on category order and cardinality, and recognize how to prevent data leakage during encoding to ensure valid model evaluation.
A churn model may need plan_tier because the observed churn rate differs for Free, Standard, and Premium subscribers. The column contains words, however, and a model needs numbers arranged as input features. A categorical variable records membership in named groups rather than a measured quantity. plan_tier is categorical because Free and Premium identify plans, not amounts we can subtract.
Encoding converts categories into numeric features while preserving the business meaning the model needs. The choice affects what a model can learn and what the resulting feature table means to us. Cardinality is the number of distinct categories in a column. plan_tier has cardinality 3, while a field such as user_id may have one distinct value per row.
Before choosing an encoding, we check two facts. We ask whether the categories have a genuine order, and we count how many categories exist. Three unordered plans call for a different representation than ratings from 1 through 5, while thousands of postal codes can make one representation impractically wide.
One-hot encoding for unordered plans
For Free, Standard, and Premium, nominal categories have names but no inherent order. Premium may cost more than Standard, but the labels alone do not state a meaningful numeric distance. One-hot encoding is usually the safe default for a small nominal column because it creates one indicator column per category.
The feature table contains columns such as plan_tier_Free, plan_tier_Standard, and plan_tier_Premium. A row for a Standard subscriber has 0, 1, and 0 in those ...