Candidate Sampling
Understand candidate sampling to optimize embedding training for large vocabularies. Learn to create weight and bias variables for efficient loss computation, improving NLP model performance and training speed.
We'll cover the following...
We'll cover the following...
Chapter Goals:
Learn about candidate sampling and why it is useful for embedding training
A. Large vocabularies
To obtain good word embeddings, it is usually necessary to train an embedding model on a large amount of text data. This means that the vocabulary size will likely be very large, often reaching tens of thousands of words. However, having a large vocabulary size can significantly slow down training.
Training an embedding model is equivalent to multiclass classification, where the possible classes include every single vocabulary word. This means that we would ...