Training Data
Understand how training data is processed for sequence to sequence models.
We'll cover the following...
Chapter Goals:
- Learn about the data used to train a seq2seq model
- Process input and output sequences into training data
A. Training task
For a seq2seq model, we use training pairs that contain an input sequence and an output sequence. For example, in machine translation the input sequence would be a sentence in one language, and the output sequence would be the sentence's correct translation in the other language.
During training, we perform two tasks:
- Input Task: Extract useful information from the input sequence
- Output Task: Calculate word probabilities at each output time step, using information from the input sequence and previous words