Transfer Learning

#machine-learning

Created on Sep 03, 2025, Last Updated on Sep 03, 2025, By a Developer

Transfer Learning refers to taking another model as a starting point for training, instead of building the model from scratch and then training it from random initialized weights.The prerequisite is that the source model for transfer learning share some level of knowledge with the task. Such as a dog recognition model can be trained on top of an object detection model.

There will be multiple reasons and cases that we would prefer transfer learning other than training from scratch,

Training data is limited for the specific task.
Saving a lot of time and computing resources, especially the source model is a large model like BERT or GPT.
Saving efforts for architecture design. However the end model will be restricted to that more or less.

Inference Head

Apply one or more layers on top of the output vector from the source model of transfer learning. In another word, the Inference Head is a slim model taken the output of source model as input, and output the desired end product. Such as performing Sentence Classification on top of BERT.

Table of Content

Inference Head

ZANE.C

Transfer Learning

Transfer Learning

Inference Head

Table of Content