Logits
Understand how to calculate logits by concatenating forward and backward BiLSTM outputs along their final dimension. Learn to extract the final time step outputs and apply a fully-connected layer to produce logits for binary text classification tasks using TensorFlow.
We'll cover the following...
Chapter Goals:
- Calculate logits from the combined BiLSTM outputs
A. Concatenation
As mentioned in the previous chapter, the BiLSTM returns two outputs: the forwards and backwards outputs. In order to calculate the model’s logits, we need to combine these two outputs. We do this through simple concatenation.
Concatenation in TensorFlow refers to appending tensors along a certain dimension. The function that performs this operation is tf.concat. It takes in two required arguments: a list of tensors to concatenate and the axis (dimension) to concatenate along.
Below we demonstrate an example usage of tf.concat. The ...