Extract the information from the big data with randomly distributed noise

by   Jin Cheng, et al.

In this manuscript, a purely data driven statistical regularization method is proposed for extracting the information from big data with randomly distributed noise. Since the variance of the noise maybe large, the method can be regarded as a general data preprocessing method in ill-posed problems, which is able to overcome the difficulty that the traditional regularization method unable to solve, and has superior advantage in computing efficiency. The unique solvability of the method is proved and a number of conditions are given to characterize the solution. The regularization parameter strategy is discussed and the rigorous upper bound estimation of confidence interval of the error in L^2 norm is established. Some numerical examples are provided to illustrate the appropriateness and effectiveness of the method.



There are no comments yet.


page 1

page 2

page 3

page 4


Identification of the Source for Full Parabolic Equations

In this work, we consider the problem of identifying the time independen...

Distributed Bayesian Matrix Decomposition for Big Data Mining and Clustering

Matrix decomposition is one of the fundamental tools to discover knowled...

A Regularization Operator for the Source Approximation of a Transport Equation

Source identification problems have multiple applications in engineering...

A Tikhonov Regularization Based Algorithm for Scattered Data with Random Noise

With the rapid growth of data, how to extract effective information from...

Learning Theory of Distributed Regression with Bias Corrected Regularization Kernel Network

Distributed learning is an effective way to analyze big data. In distrib...

Recognizing the Tractability in Big Data Computing

Due to the limitation on computational power of existing computers, the ...

A variational non-linear constrained model for the inversion of FDEM data

Reconstructing the structure of the soil using non invasive techniques i...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.