Understanding CNNs: Fully Connected Layers

Fully connected layers

Fully connected layers are a fully connected set of weights from the input to the output. These fully connected weights are able to learn global information as they are connected from each input to each output. Also, having such layers of full connectedness allows us to combine features learned by the convolution layers preceding the fully connected layers globally to produce meaningful outputs.

Let’s define the output of the last convolution or pooling layer to be of size $p\times o \times d$, where $p$ is the height of the input, $o$ is the width of the input, and $d$ is the depth of the input. As an example, think of an RGB image, which will have a fixed height, fixed width, and a depth of 3 (one depth channel for each RGB component).

Then, for the initial fully connected layer found immediately after the last convolution or pooling layer, the weight matrix will be $w^{(m, \space p\times o\times d)}$, where $height \times width \times depth$ of the layer output is the number of output units produced by that last layer, and $m$ is the number of hidden units in the fully connected layer. Then, during inference (or prediction), we reshape the output of the last convolution/pooling layer to be of size $(p \times o \times d, 1)$and perform the following matrix multiplication to obtain $h$: