Transfer Learning for Custom Model Creation

Complex DL models require large datasets with millions of training data points. Training a complex DL model on a small amount of data leads to model overfitting. When we don’t have access to a large dataset, we resort to transfer learning, which uses the knowledge gained (learned network parameters) from existing state-of-the-art models. Let’s implement transfer learning of a large pretrained model to solve a new task.

Transfer of knowledge for DL

Training large-scale datasets, such as the 1,000-class ILSVRC dataset, requires huge memory and computational resources. Transfer learning allows us to reuse knowledge and pretrained network parameters from an existing model, which has been trained on a large dataset, and apply it to a new problem or task. Because of transfer learning, researchers and practitioners don’t train networks from scratch for their particular problem.

Transfer learning uses trained parameters to solve a different but related problem. A car recognition model, for instance, can be applied to recognize buses and trucks. The following figure shows the dashed arrows from the top network to the bottom network to indicate the transfer of learned network parameters. Usually, fully connected layers on the output side are redefined and are trained to solve the new problem using transfer learning.