Building Advanced Deep Learning and NLP Projects/

...

/

Project Creation: Part Three

Project Creation: Part Three

In this lesson, we will be creating the model and training it on our dataset.

We'll cover the following...

In the previous lesson, we discussed a lot of topics. Now we will be moving ahead and building our model architecture.

Model architecture

We will define a function that will accept four parameters: the input shape, the output sequence length, the vocabulary size of encrypted text, and the vocabulary size of the plaintext.

Press + to interact
Python 3.5
def simple_model(input_shape, output_sequence_length, code_vocab_size, plaintext_vocab_size):
    
    learning_rate = 1e-3
    input_seq = Input(input_shape[1:])
    rnn = GRU(64, return_sequences=True)(input_seq)
    logits = TimeDistributed(Dense(plaintext_vocab_size))(rnn)
    model = Model(input_seq, Activation('softmax')(logits))
    model.compile(loss=sparse_categorical_crossentropy,
                  optimizer=Adam(learning_rate),
                  metrics=['accuracy'])
    return model
print("Function Created Successfully!")

Explanation:

  • We first defined our function, which accepts the four parameters which we discussed above.
  • On line 3, we defined our learning rate for the network.
  • On line 5, we created an Input() object by specifying the shape of the input data as a parameter.
  • On line 6, we defined our GRU layer with 64 units. This means that RNN will be unrolled 64 times. Also, the return_sequences=True specifies whether to return the last output in the output sequence or not.
  • On line 7, we defined our TimeDistributed layer by passing a dense layer as its parameter.
  • We also specified input_seq and rnn after the function
...