Dockerfile.tar.gz

Numerical Variables

Default Job

In this course, you will learn how to apply basic and more advanced feature engineering to tabular data with python, during this course,  we will see a range of different techniques and methods to handle many common cases within the data set, so in result, you will create great features so that your machine learning models can predict good results.

Feature Engineering with Python

## Categorical variable — rare labels

Different labels appear in a variable with different frequencies. Rare labels have a shallow frequency of occurrence compared to other labels. 

For example, for the variable city where a French citizen lives:
- Paris is a frequent category (many people live in Paris)
- Molring, Moselle is a rare category (not many people live there)

## Impact of Rare Labels
Rare labels can cause some impacts on the model. Here is why:
- Rare labels may lead to overfitting and generalization problems.
It is hard to understand the rare label's role in the final prediction.
- Eliminating rare labels may enhance model performance.

We can visualize the labels of the variables using count plots, as we saw previously. 

The following plot describes the different movie categories:



# Categorical variable — rare labels

Different labels appear in a variable with different frequencies. Rare labels have a shallow frequency of occurrence compared to other labels. 

For example, for the variable city where a French citizen lives:
- Paris is a frequent category (many people live in Paris)
- Molring, Moselle is a rare category (not many people live there)

# Impact of Rare Labels
Rare labels can cause some impacts on the model. Here is why:
- Rare labels may lead to overfitting and generalization problems.
It is hard to understand the rare label's role in the final prediction.
- Eliminating rare labels may enhance model performance.

We can visualize the labels of the variables using count plots, as we saw previously. 

The following plot describes the different movie categories:



Categorical variables have "rare labels," which is one of the problems we often encounter in datasets. In this lesson, we will cover the issue of rare labels, how to find them in our data, and why they should be processed.


Categorical variable — rare labels

Categorical variables have "rare labels," which is one of the problems we often encounter in datasets. In this lesson, we will cover the issue of rare labels, how to find them in our data, and why they should be processed.

Introduction

Variable Types

Common Concerns in Datasets

Handling & Imputing Missing Values

Encoding Categorical Variables

Transforming Variables

Variable Discretization

Handling Outliers

Feature Scaling

Engineering Geospatial Data

Handling Date-Time and Mixed Variables

Resampling Imbalanced Data

Advanced Feature Engineering Techniques

Conclusion

Categorical variable — rare labels

Categorical variable — rare labels