Optimization with Gradient Descent

In the previous lesson, we looked at the intuition behind the gradient descent algorithm and the update equation. In this lesson, we are going to implement it in Python. We are going to predict the tips paid by a customer at a restaurant. We will choose the best model using gradient descent.

Minimization with Gradient Descent

Recall that the gradient descent algorithm is

• Start with a random initial value of $\theta$.
• Compute $\theta_t - \alpha \frac{\partial}{\partial \theta} L(\theta_t,Y)$ to update the value of $\theta$.
• Keep updating the value of $\theta$ until it stops changing values. This can be the point where we have reached the minimum of the error function.

We will be using the tips dataset that has the following data.