The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design

11/13/2019
by   Jeffrey Dean, et al.
92

The past decade has seen a remarkable series of advances in machine learning, and in particular deep learning approaches based on artificial neural networks, to improve our abilities to build more accurate systems across a broad range of areas, including computer vision, speech recognition, language translation, and natural language understanding tasks. This paper is a companion paper to a keynote talk at the 2020 International Solid-State Circuits Conference (ISSCC) discussing some of the advances in machine learning, and their implications on the kinds of computational devices we need to build, especially in the post-Moore's Law-era. It also discusses some of the ways that machine learning may also be able to help with some aspects of the circuit design process. Finally, it provides a sketch of at least one interesting direction towards much larger-scale multi-task models that are sparsely activated and employ much more dynamic, example- and task-based routing than the machine learning models of today.

READ FULL TEXT

page 4

page 7

page 8

page 12

research
06/15/2017

An Overview of Multi-Task Learning in Deep Neural Networks

Multi-task learning (MTL) has led to successes in many applications of m...
research
06/21/2022

Why Robust Natural Language Understanding is a Challenge

With the proliferation of Deep Machine Learning into real-life applicati...
research
01/02/2017

An affective computational model for machine consciousness

In the past, several models of consciousness have become popular and hav...
research
10/31/2022

A Machine Learning Tutorial for Operational Meteorology, Part II: Neural Networks and Deep Learning

Over the past decade the use of machine learning in meteorology has grow...
research
09/04/2022

A Review of Sparse Expert Models in Deep Learning

Sparse expert models are a thirty-year old concept re-emerging as a popu...
research
12/11/2020

The Rise of AI-Driven Simulators: Building a New Crystal Ball

The use of computational simulation is by now so pervasive in society th...
research
03/06/2018

Learning Memory Access Patterns

The explosion in workload complexity and the recent slow-down in Moore's...

Please sign up or login with your details

Forgot password? Click here to reset