Search⌘ K
Home
Courses
Natural Language Processing with Machine Learning

RNN/LSTM

Explore the concepts behind recurrent neural networks and long short-term memory cells, understanding how they use sequential data and gates to capture dependencies in text. This lesson helps you grasp LSTM architectures and their advantage over regular RNNs, enabling better modeling of long-range word relationships for tasks like language prediction.

We'll cover the following...

Chapter Goals:

  • Understand the differences between feed-forward and recurrent neural networks

  • Learn about the long short-term memory (LSTM) cell

A. Neural network types

As mentioned in the previous chapter, the MLP is a type of feed-forward neural network. This means that each of its layers is a fixed size, and each layer's output is fed into the next layer as input. While feed-forward neural networks are great for tasks involving fixed size input data, they aren't as great in dealing with sequences of text data.

For the remainder of this course, we'll be focusing on recurrent neural networks, which are specially designed to work with sequential data of varying lengths. The main component of a recurrent neural network (RNN) is its cell.

Diagram of an RNN. On the left is the rolled RNN, on the right is the unrolled RNN with 3 time steps. The square represents the RNN cell.
Diagram of an RNN. On the left is the rolled RNN, on the right is the unrolled RNN with 3 time steps. The square represents the RNN cell.

The rolled RNN diagram shows the "true" depiction of the network. It consists of a single cell (although a multi-layer RNN will have multiple stacked cells), and 3 types of connections: ...