Classifying Documents with Document Embeddings
Explore how to build a multiclass logistic regression model using document embeddings created from word vector algorithms such as skip-gram, CBOW, GloVe, and ELMo. Gain insight into the performance differences of these embeddings for classifying BBC news articles into categories like entertainment, technology, politics, business, and sports.
We'll cover the following...
We’ll be training a simple multiclass (or a multinomial) logistic regression classifier on this data. The logistic regression model will look as follows:
The image above depicts the multinomial logistic regression model. The model takes in an embedding vector and outputs a probability distribution over different available classes.
It’s a very simple model with a single layer, where the input is the embedding vector (e.g., a 128-element-long vector), and the output is a five-node softmax layer that will output the likelihood of the input belonging to each category as a probability distribution.
We’ll be training several models as opposed to a single run. This will give us a more consistent result ...