Dockerfile.tar.gz

Numerical Variables

Default Job

In this course, you will learn how to apply basic and more advanced feature engineering to tabular data with python, during this course,  we will see a range of different techniques and methods to handle many common cases within the data set, so in result, you will create great features so that your machine learning models can predict good results.

Feature Engineering with Python

## Cyclical feature problem
Date and time variables have an essential characteristic called **periodicity**. 

For example: if we take the 23:15 and 04:15 times in a 24h format, which one is the closest to 00:15? The answer is obviously 23:15, with a difference of only 1 hour.

However, machine learning models will answer this question based on basic numeric terms, so our model will assume that 4:15 is closer. As feature engineers, it is our job to fix this problem.

Hours, minutes, seconds, days of the month, days of weeks, and months are all cyclical or periodic features that follow a cycle. We definitely need to save this information in our datasets to help our model learn accurately and behave correctly.

The way to realize that is to project the cyclical feature on the unit circle, which will invite the sin and cos functions to join our journey in creating expressive features. After projecting the data, we create two new features: the sin and cos transformation of the cyclical feature. So, each feature will be represented by two coordinates. 

See the illustration below:



# Cyclical feature problem
Date and time variables have an essential characteristic called **periodicity**. 

For example: if we take the 23:15 and 04:15 times in a 24h format, which one is the closest to 00:15? The answer is obviously 23:15, with a difference of only 1 hour.

However, machine learning models will answer this question based on basic numeric terms, so our model will assume that 4:15 is closer. As feature engineers, it is our job to fix this problem.

Hours, minutes, seconds, days of the month, days of weeks, and months are all cyclical or periodic features that follow a cycle. We definitely need to save this information in our datasets to help our model learn accurately and behave correctly.

The way to realize that is to project the cyclical feature on the unit circle, which will invite the sin and cos functions to join our journey in creating expressive features. After projecting the data, we create two new features: the sin and cos transformation of the cyclical feature. So, each feature will be represented by two coordinates. 

See the illustration below:



We will learn about a unique characteristic of time and date variables called periodicity during this lesson. We will look at a good example, a detailed illustration, and a code sample to get that done in practice.

Introduction

Variable Types

Common Concerns in Datasets

Handling & Imputing Missing Values

Encoding Categorical Variables

Transforming Variables

Variable Discretization

Handling Outliers

Feature Scaling

Engineering Geospatial Data

Handling Date-Time and Mixed Variables

Resampling Imbalanced Data

Advanced Feature Engineering Techniques

Conclusion

Cyclical feature problem

Cyclical feature problem