Search⌘ K
Home
Courses
Natural Language Processing with TensorFlow

Training the NMT

Understand the process of training neural machine translation models by preparing data, implementing custom training loops in TensorFlow, and tracking performance using the BLEU score, a key metric for evaluating translation quality in NLP.

We'll cover the following...

Now that we have defined the NMT architecture and preprocessed the training data, it’s quite straightforward to train the model. Here, we’ll define and illustrate the exact process used for training:

The training procedure for NMT
The training procedure for NMT

The prepare_data() function

For the model training, we’re going to define a custom training loop because there’s a special metric we’d like to track. Unfortunately, this metric is not a readily available TensorFlow metric. But before that, there are several utility functions we need to define:

Python 3.10.4
def prepare_data(de_lookup_layer, train_xy, valid_xy, test_xy):
    """ Create a data dictionary from the DataFrames containing data
    """
    data_dict = {}
    for label, data_xy in zip(['train', 'valid', 'test'], [train_xy, valid_xy, test_xy]):
        data_x, data_y = data_xy
        en_inputs = data_x
        de_inputs = data_y[:,:-1]
        de_labels = de_lookup_layer(data_y[:,1:]).numpy()
        data_dict[label] = {'encoder_inputs': en_inputs, 'decoder_inputs': de_inputs, 'decoder_labels': de_labels}
    return data_dict

The prepare_data() function takes the source sentence and target sentence pairs and generates encoder and decoder inputs and decoder labels. Let’s look at the arguments:

  • de_lookup_layer: The StringLookup layer of the German language.

  • train_xy: A tuple containing tokenized English sentences and tokenized German sentences in the training set, respectively.

  • valid_xy: Similar to train_xy but for validation data.

  • test_xy: Similar to train_xy but for test data.

For each training, validation, and test dataset, this function generates the following: ...