Univariate Linear Regression

Univariate linear regression

In univariate linear regression, we have one independent variable $x$, which we use to predict a dependent variable $y$.

We’ll use the tips dataset from seaborn’s datasets to illustrate theoretical concepts.

---

---

We’ll use the following columns from the dataset for univariate analysis:

• Total_bill: It is the total bill of food served.

• Tip: It is the tip given on the cost of food.

---

Goal of univariate linear regression: The goal is to predict the “tip” given on a “total_bill”. The regression model constructs an equation to do so.

If we plot the scatter plot between the independent variable (total_bill) and the dependent variable (tip), we’ll get the plot below:

---

---

• We can see that the points in the scatter plot are mostly scattered along the diagonal.

• This indicates that there may be a positive correlation between the total_bill and tip. This will be useful for modeling.

---

Working

The univariate linear regression model comes up with the following equation of the straight line:

$\hat{y} = w_0 + w_1 * x$

Or

$tip\_predicted = w_0 + w_1 * total\_bill$

Goal: Find the values of $w_0$ and $w_1$, where $w_0$ and $w_1$ are the parameters, so that the predicted tip ($\hat{y}$) is as close to the actual tip ($y$) as possible. Mathematically, we can model the problem as seen below.

$J(w_0, w_1)$ = $\frac{1}{2m}\sum_{i=1}^{m}(\hat{y}^i-y^i)^2$

• $J(w_0, w_1)$ is the cost function, which an algorithm tries to minimize by finding the values of $w_0$ and $w_1$. These values give us the minimum value of the above function.

• $y^i$ is the actual output value of a training instance $i$, where $i =1,2,3 ..$

• $\hat{y}^i$ ...