A Recursive Neural Network (RNN) falls under the category of a deep neural network. It is constructed in such a way that it includes applying the same set of weights
A simple RNN structure is illustrated in the figure below:
A recursive neural network uses the learning algorithms, which means it predicts the output values (
Tree RNNs are commonly used to implement the
score which is calculated at each traversal of nodes that tells us which pair of phrases and words need to be combined first to form the best syntactic tree which explains a given sentence. The question arises, how do we actually calculate the score and merge the phrases/inputs? Here's an architecture of simple Tree-RNN through which we will explain the working of a recursive neural network.
In our illustration, x1 and x2 are carrying the words. At the hidden layer, these words are merging in the form of vector representation p, which is calculated through the following recurrence relation.
After the merged pair calculation, we need to find the score S at each node which is calculated by the following relation:
We will get S by multiplying particular weight W and merging vector P. This enables us to find the best pair of
The above illustration shows that there are different words at each input, and they are combined together at the hidden layer. This procedure is done by calculating P and S. The traversing is done at each node and the score is calculated recursively. In the end, all the words are merged at the top and form a sentence which in our example is "A small crowd quietly enters the historic church."
# initialize the recursive neural network and the hidden layers my_rnn = RNN() hidden_layers = [0, 0, 0, 0, 0, 0, 0, 0] sequence = ["A", "small", "crowd", "quietly","enters","the","historic","church"] # feeding the sequence as input to the network for word in sequence: prediction, hidden_layers = my_rnn(word, hidden_layers) # prediction for the next word next_word = prediction
Therefore, a recursive neural network is a tree-based structure that uses the learning algorithms for
View all Courses