DeepAI AI Chat
Log In Sign Up

Accurate, Fast and Scalable Kernel Ridge Regression on Parallel and Distributed Systems

by   Yang You, et al.
University of California-Davis
Georgia Institute of Technology
berkeley college

We propose two new methods to address the weak scaling problems of KRR: the Balanced KRR (BKRR) and K-means KRR (KKRR). These methods consider alternative ways to partition the input dataset into p different parts, generating p different models, and then selecting the best model among them. Compared to a conventional implementation, KKRR2 (optimized version of KKRR) improves the weak scaling efficiency from 0.32 getting the same accuracy by using the same data and the same hardware (1536 processors). BKRR2 (optimized version of BKRR) achieves a higher accuracy than the current fastest method using less training time for a variety of datasets. For the applications requiring only approximate solutions, BKRR2 improves the weak scaling efficiency to 92 speedup: 4096 times).


page 1

page 2

page 3

page 4


An argument in favor of strong scaling for deep neural networks with small datasets

In recent years, with the popularization of deep learning frameworks and...

Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism

Scaling CNN training is necessary to keep up with growing datasets and r...

Weak Scaling of DSA Preconditioning of Transport Sweeps using HYPRE

This report summarizes the weak scaling performance of the diffusion-syn...

ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization

We propose a multitask pretraining approach ZeroPrompt for zero-shot gen...

Weak Evolvability Equals Strong Evolvability

An updated version will be uploaded later....

A High-Performance Implementation of a Robust Preconditioner for Heterogeneous Problems

We present an efficient implementation of the highly robust and scalable...

Scalable Breadth-First Search on a GPU Cluster

On a GPU cluster, the ratio of high computing power to communication ban...