Bidirectional LSTM
Explore how bidirectional LSTMs enhance text classification by processing sequences forward and backward, allowing better context understanding. Learn to implement BiLSTM using TensorFlow's keras Bidirectional layer and manage LSTM outputs for NLP tasks.
We'll cover the following...
Chapter Goals:
Learn about the bidirectional LSTM and why it's used
A. Forwards and backwards
The language model from the Language Model section of this course used a regular LSTM which read each input sequence in the forwards direction. This meant that the recurrent connections went in the left-right direction, i.e. from time step ttt to time step t+1t + 1t+1.
While regular LSTMs work well for most NLP tasks, they are not always the best option. Specifically, when we have access to a completed text sequence (e.g. text classification), it may be beneficial to look at the sequence in both the forwards and backwards directions.
By looking at a sequence in both directions, the model can take into account both the past and future context of words and phrases in the text sequence, which allows it to obtain a better understanding of the text sequence.
B. Bidirectional LSTM structure
The way we look at an input sequence in both directions is with a model called a bidirectional LSTM, or BiLSTM for short. The model architecture is incredibly simple: it consists of a regular forwards LSTM and a backwards LSTM, which reads the input sequence in reverse.
The above diagram shows a ...