Parallelization of Machine Learning Algorithms Respectively on Single Machine and Spark

05/08/2022
by   Jiajun Shen, et al.
0

With the rapid development of big data technologies, how to dig out useful information from massive data becomes an essential problem. However, using machine learning algorithms to analyze large data may be time-consuming and inefficient on the traditional single machine. To solve these problems, this paper has made some research on the parallelization of several classic machine learning algorithms respectively on the single machine and the big data platform Spark. We compare the runtime and efficiency of traditional machine learning algorithms with parallelized machine learning algorithms respectively on the single machine and Spark platform. The research results have shown significant improvement in runtime and efficiency of parallelized machine learning algorithms.

READ FULL TEXT
research
01/12/2022

Detection of brain tumors using machine learning algorithms

An algorithm capable of processing NMR images was developed for analysis...
research
02/08/2018

Deep Learning with Apache SystemML

Enterprises operate large data lakes using Hadoop and Spark frameworks t...
research
11/17/2022

Air pollution models in epidemiologic studies with geostatistics and machine learning

Development of air pollution models for large regions is a priority for ...
research
10/08/2018

Effective Parallelisation for Machine Learning

We present a novel parallelisation scheme that simplifies the adaptation...
research
07/06/2022

WE model: A Machine Learning Model Based on Data-Driven Movie Derivatives Market Prediction

The mature development and the extension of the industry chain make the ...
research
06/01/2020

When Machine Learning Meets Multiscale Modeling in Chemical Reactions

Due to the intrinsic complexity and nonlinearity of chemical reactions, ...
research
06/19/2020

Pervasive Lying Posture Tracking

There exist significant gaps in research about how to design efficient i...

Please sign up or login with your details

Forgot password? Click here to reset