DeepAI AI Chat
Log In Sign Up

A General Theory for Training Learning Machine

by   Hong Zhao, et al.

Though the deep learning is pushing the machine learning to a new stage, basic theories of machine learning are still limited. The principle of learning, the role of the a prior knowledge, the role of neuron bias, and the basis for choosing neural transfer function and cost function, etc., are still far from clear. In this paper, we present a general theoretical framework for machine learning. We classify the prior knowledge into common and problem-dependent parts, and consider that the aim of learning is to maximally incorporate them. The principle we suggested for maximizing the former is the design risk minimization principle, while the neural transfer function, the cost function, as well as pretreatment of samples, are endowed with the role for maximizing the latter. The role of the neuron bias is explained from a different angle. We develop a Monte Carlo algorithm to establish the input-output responses, and we control the input-output sensitivity of a learning machine by controlling that of individual neurons. Applications of function approaching and smoothing, pattern recognition and classification, are provided to illustrate how to train general learning machines based on our theory and algorithm. Our method may in addition induce new applications, such as the transductive inference.


page 1

page 2

page 3

page 4


A New Type of Neurons for Machine Learning

In machine learning, the use of an artificial neural network is the main...

General Vector Machine

The support vector machine (SVM) is an important class of learning machi...

Amended Cross Entropy Cost: Framework For Explicit Diversity Encouragement

Cross Entropy (CE) has an important role in machine learning and, in par...

Data-Driven Optimized Tracking Control Heuristic for MIMO Structures: A Balance System Case Study

A data-driven computational heuristic is proposed to control MIMO system...

L_1-regularized Boltzmann machine learning using majorizer minimization

We propose an inference method to estimate sparse interactions and biase...

Bridging belief function theory to modern machine learning

Machine learning is a quickly evolving field which now looks really diff...