What is Transfer Learning?
Transfer or inductive learning is a supervised learning technique that reuses parts of a previously trained model on a new network tasked for a different but similar problem. In computer vision, for example, some feature extractors from a nudity detection model could be used to speed up the learning process for a new facial recognition model.
How is Transfer Learning Used?
Using a pre-trained model significantly reduces the time required for feature engineering and training. The first step is to select a source model, ideally one with a large dataset to train with. Many research institutions release these models and datasets as open-sourced projects, so it’s not necessary to create your own.
The next step is to decide which layers to reuse in your own network. The goal is create a framework that is at least better than a naïve model, so you can be assured some new feature learning takes place. Typically deeper layers are reused as these tend to be more general whereas the top layers tend to more finely tuned to a particular problem.
Finally, the new model is trained on the new data set as usual. The advantage is that the model tends to convergence much faster and thus less data and compute time is required.
