Review of Machine Learning

Overview

Machine learning is a set of techniques that allows computer programs to optimally perform a given task based on observed data. While classical AI techniques relied on painstakingly describing step-by-step behaviors and trees of logic, machine learning provides machinery that allows a program to learn behavior from inputs and outputs. As such, high-quality data is extremely important for good machine learning, perhaps the most important thing.

In addition to good data, machine learning requires a loss function, a model, and an optimization strategy. We’ll provide the briefest overview possible of each of these components.

Data

Machine learning can be divided into two camps of techniques: supervised learning and unsupervised learning. The difference between the two is basically determined by what we wish to accomplish with our data.

Note: In reality, there are many other techniques that exist on a spectrum between unsupervised and supervised learning, such as semi-supervised learning, self-supervised learning, and reinforcement learning.

Supervised learning

In supervised learning, data consists of labeled pairs $(x, y)$, where $x$ is an observed data point (for instance, an x-y-z coordinate in 3D space) from a population $X$ and $y$ is some observed value (for instance, the luminance value emitted from that coordinate in 3D space). The objective is to learn some function $y = f(x)$ that maps from an input observation $x$ to a predicted $y$ value. In practice, this function $f$ also consists of a set of learnable parameters ...