Neural Machine Translation
Explore how to build transformer-based neural machine translation models that correct grammatical errors by translating incorrect sentences into correct ones. Understand dataset preparation, error annotation, loss functions, and evaluation metrics necessary to train and deploy effective grammar correction systems.
We'll cover the following...
Deep models for grammar correction
Modern grammar correction methods are mainly transformer-based neural machine translation (NMT) models. For spelling correction, transformers can be studied as a machine translation problem, where instead of translating from a source language to a target language, we can use misspelled words as the "source language”, and correctly spelled words in our language of choice as the "target language". For grammar correction, we can modify this architecture, instead translating a sentence from a grammatically incorrect sentence as our "source language" and a grammatically correct sentence as our "target language". Since these problems are quite similar, many modern methods combine these two to create a single transformer that can handle both spelling and grammar correction.
Training an NMT model for grammar correction
Training an NMT model follows a fairly typical encoder-decoder architecture, with some slight deviations around error annotation and data preparation, as well as customizing the ...