Knowledge Transfer Between Artificial Intelligence Systems

09/05/2017
by   Ivan Y. Tyukin, et al.
0

We consider the fundamental question: how a legacy "student" Artificial Intelligent (AI) system could learn from a legacy "teacher" AI system or a human expert without complete re-training and, most importantly, without requiring significant computational resources. Here "learning" is understood as an ability of one system to mimic responses of the other and vice-versa. We call such learning an Artificial Intelligence knowledge transfer. We show that if internal variables of the "student" Artificial Intelligent system have the structure of an n-dimensional topological vector space and n is sufficiently high then, with probability close to one, the required knowledge transfer can be implemented by simple cascades of linear functionals. In particular, for n sufficiently large, with probability close to one, the "student" system can successfully and non-iteratively learn k≪ n new examples from the "teacher" (or correct the same number of mistakes) at the cost of two additional inner products. The concept is illustrated with an example of knowledge transfer from a pre-trained convolutional neural network to a simple linear classifier with HOG features.

READ FULL TEXT
research
12/14/2017

Three IQs of AI Systems and their Testing Methods

The rapid development of artificial intelligence has brought the artific...
research
03/13/2021

Student-Teacher Learning from Clean Inputs to Noisy Inputs

Feature-based student-teacher learning, a training method that encourage...
research
07/09/2021

Lifelong Teacher-Student Network Learning

A unique cognitive capability of humans consists in their ability to acq...
research
10/03/2016

One-Trial Correction of Legacy AI Systems and Stochastic Separation Theorems

We consider the problem of efficient "on the fly" tuning of existing, or...
research
02/09/2021

NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application

Pre-trained language models (PLMs) like BERT have made great progress in...
research
08/28/2020

A Metamodel and Framework for AGI

Can artificial intelligence systems exhibit superhuman general intellige...
research
11/22/2019

Knowledge Network and a Knowledge Network Example

Knowledge networks can be defined as social networks that enable the tra...

Please sign up or login with your details

Forgot password? Click here to reset