0% completed

All LessonsFree Lessons (7)

What you'll learn from this course

Word Embeddings

Introduction Vocabulary Embeddings Skip-gram Embedding Matrix Candidate Sampling Embedding Loss Cosine Similarity K-Nearest Neighbors Quiz

Language Model

Introduction Language Model Padding RNN/LSTM Dropout Multiple Layers LSTM Output Calculating Loss Predictions Tensor Indexing Quiz

Text Classification

Introduction Sentiment Analysis Embeddings Bidirectional LSTM Logits Loss Classification Improving the Model Quiz

Seq2Seq Model

Encoder-Decoder

Training Sampler

Decoding Output

Calculating Loss

Inference Decoding

Model Improvement

Course Conclusion

Fundamentals of NLP

Natural Language Processing with Machine Learning/

...

/

Combined State

Combined State

Combine the final states for a BiLSTM into usable initial states.

We'll cover the following...

Chapter Goals:

Combine the final states for each BiLSTM layer

A. LSTMStateTuple initialization

We initialize an LSTMStateTuple object with a hidden state (c) and state output (h).

Below we show an example of initializing an LSTMStateTuple object using the final forward and backward states from a single layer BiLSTM encoder.

Python 3.5

import tensorflow as tf
# Forward state of single-layer BiLSTM final states
fw_c = forward_final_state[0]
fw_h = forward_final_state[1]
# Backward state of single-layer BiLSTM final states
bw_c = backward_final_state[0]
bw_h = backward_final_state[1]
# Concatenate along final axis
final_c = tf.concat([fw_c, bw_c], -1 )
final_h = tf.concat([fw_h, bw_h], -1)
combined_state = tf.compat.v1.nn.rnn_cell.LSTMStateTuple(
    final_c, final_h)
print(combined_state)

In the above example, we combined the BiLSTM forward and ...