The Continuous Bag-of-Words Algorithm
Explore the Continuous Bag-of-Words (CBOW) model to understand how it predicts a target word from surrounding context words. This lesson covers data preparation, model architecture including embedding layers, training with TensorFlow, and evaluation through word vector similarities, helping you implement CBOW effectively.
How the CBOW model works
The continuous bag-of-words (CBOW) model works in a similar way to the skip-gram algorithm, with one significant change in the problem formulation. In the skip-gram model, we predict the context words from the target word. However, in the CBOW model, we predict the target word from contextual words. Let’s compare what data looks like for the skip-gram algorithm and the CBOW model by taking the example sentence:
The dog barked at the mailman.
For the skip-gram algorithm, the data tuples—(input word, output word)—might look like this:
(dog, the), (dog, barked), (barked, dog), and so on.
For CBOW, the data tuples would look like the following:
([the, barked], dog), ([dog, at], barked), and so on.
Consequently, the input of the CBOW has a dimensionality of where is the context window size, and ...