Implementing the Skip-Gram Architecture with TensorFlow

We’ll now walk through an implementation of the skip-gram algorithm that uses the TensorFlow library.

Defining hyperparameters

First, let’s define the hyperparameters of the model. We’re free to change these hyperparameters to see how they affect final performance (for example, batch_size = 1024 or batch_size = 2048). However, since this is a simpler problem than the more complex real-world problems, we might not see any significant differences (unless we change them to extremes, for example, batch_size = 1 or num_sampled = 1):