Song Han

is this you? claim profile

∙ 0 followers
MIT ∙ University of Connecticut ∙ Stanford University

Assistant professor in the Electrical Engineering and Computer Science Department of the Massachusetts Institute of Technology (MIT).

SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning

The attention mechanism is becoming increasingly popular in Natural Lang...
12/17/2020 ∙ by Hanrui Wang, et al. ∙ 0 ∙ share

read it
IOS: Inter-Operator Scheduler for CNN Acceleration

To accelerate CNN inference, existing deep learning frameworks focus on ...
11/02/2020 ∙ by Yaoyao Ding, et al. ∙ 0 ∙ share

read it
Hardware-Centric AutoML for Mixed-Precision Quantization

Model quantization is a widely used technique to compress and accelerate...
08/11/2020 ∙ by Kuan Wang, et al. ∙ 7 ∙ share

read it
Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution

Self-driving cars need to understand 3D scenes efficiently and accuratel...
07/31/2020 ∙ by Haotian Tang, et al. ∙ 4 ∙ share

read it
Tiny Transfer Learning: Towards Memory-Efficient On-Device Learning

We present Tiny-Transfer-Learning (TinyTL), an efficient on-device learn...
07/22/2020 ∙ by Han Cai, et al. ∙ 21 ∙ share

read it
MCUNet: Tiny Deep Learning on IoT Devices

Machine learning on tiny IoT devices based on microcontroller units (MCU...
07/20/2020 ∙ by Ji Lin, et al. ∙ 0 ∙ share

read it
Differentiable Augmentation for Data-Efficient GAN Training

The performance of generative adversarial networks (GANs) heavily deteri...
06/18/2020 ∙ by Shengyu Zhao, et al. ∙ 11 ∙ share

read it
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy

We present APQ for efficient deep learning inference on resource-constra...
06/15/2020 ∙ by Tianzhe Wang, et al. ∙ 0 ∙ share

read it
HAT: Hardware-Aware Transformers for Efficient Natural Language Processing

Transformers are ubiquitous in Natural Language Processing (NLP) tasks, ...
05/28/2020 ∙ by Hanrui Wang, et al. ∙ 24 ∙ share

read it
MicroNet for Efficient Language Modeling

It is important to design compact language models for efficient deployme...
05/16/2020 ∙ by Zhongxia Yan, et al. ∙ 0 ∙ share

read it
GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning

Automatic transistor sizing is a challenging problem in circuit design d...
04/30/2020 ∙ by Hanrui Wang, et al. ∙ 0 ∙ share

read it
Lite Transformer with Long-Short Range Attention

Transformer has become ubiquitous in natural language processing (e.g., ...
04/24/2020 ∙ by Zhanghao Wu, et al. ∙ 0 ∙ share

read it
A Fast Algorithm for Source-Wise Round-Trip Spanners

In this paper, we study the problem of efficiently constructing source-w...
04/12/2020 ∙ by Chun Jiang Zhu, et al. ∙ 0 ∙ share

read it
GAN Compression: Efficient Architectures for Interactive Conditional GANs

Conditional Generative Adversarial Networks (cGANs) have enabled control...
03/19/2020 ∙ by Muyang Li, et al. ∙ 69 ∙ share

read it
SpArch: Efficient Architecture for Sparse Matrix Multiplication

Generalized Sparse Matrix-Matrix Multiplication (SpGEMM) is a ubiquitous...
02/20/2020 ∙ by Zhekai Zhang, et al. ∙ 0 ∙ share

read it
ChainSplitter: Towards Blockchain-based Industrial IoT Architecture for Supporting Hierarchical Storage

The fast developing Industrial Internet of Things (IIoT) technologies pr...
10/02/2019 ∙ by Gang Wang, et al. ∙ 0 ∙ share

read it
Training Kinetics in 15 Minutes: Large-scale Distributed Training on Videos

Deep video recognition is more computationally expensive than image reco...
10/01/2019 ∙ by Ji Lin, et al. ∙ 0 ∙ share

read it
Once for All: Train One Network and Specialize it for Efficient Deployment

Efficient deployment of deep learning models requires specialized neural...
08/26/2019 ∙ by Han Cai, et al. ∙ 0 ∙ share

read it
Point-Voxel CNN for Efficient 3D Deep Learning

We present Point-Voxel CNN (PVCNN) for efficient, fast 3D deep learning....
07/08/2019 ∙ by Zhijian Liu, et al. ∙ 0 ∙ share

read it
Deep Leakage from Gradients

Exchanging gradients is a widely used method in modern multi-node machin...
06/21/2019 ∙ by Ligeng Zhu, et al. ∙ 2 ∙ share

read it
Design Automation for Efficient Deep Learning Computing

Efficient deep learning computing requires algorithm and hardware co-des...
04/24/2019 ∙ by Song Han, et al. ∙ 0 ∙ share

read it
Defensive Quantization: When Efficiency Meets Robustness

Neural network quantization is becoming an industry standard to efficien...
04/17/2019 ∙ by Ji Lin, et al. ∙ 0 ∙ share

read it
SysML: The New Frontier of Machine Learning Systems

Machine learning (ML) techniques are enjoying rapidly increasing adoptio...
03/29/2019 ∙ by Alexander Ratner, et al. ∙ 0 ∙ share

read it
Fully Distributed Packet Scheduling Framework for Handling Disturbances in Lossy Real-Time Wireless Networks

Along with the rapid growth of Industrial Internet-of-Things (IIoT) appl...
02/06/2019 ∙ by Tianyu Zhang, et al. ∙ 0 ∙ share

read it
Learning to Design Circuits

Analog IC design relies on human experts to search for parameters that s...
12/05/2018 ∙ by Hanrui Wang, et al. ∙ 0 ∙ share

read it
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

Neural architecture search (NAS) has a great impact by automatically des...
12/02/2018 ∙ by Han Cai, et al. ∙ 0 ∙ share

read it
HAQ: Hardware-Aware Automated Quantization

Model quantization is a widely used technique to compress and accelerate...
11/21/2018 ∙ by Kuan Wang, et al. ∙ 0 ∙ share

read it
Temporal Shift Module for Efficient Video Understanding

The explosive growth in online video streaming gives rise to challenges ...
11/20/2018 ∙ by Ji Lin, et al. ∙ 8 ∙ share

read it
Communication-Optimal Distributed Dynamic Graph Clustering

We consider the problem of clustering graph nodes over large-scale dynam...
11/14/2018 ∙ by Chun Jiang Zhu, et al. ∙ 0 ∙ share

read it
Path-Level Network Transformation for Efficient Architecture Search

We introduce a new function-preserving transformation for efficient neur...
06/07/2018 ∙ by Han Cai, et al. ∙ 2 ∙ share

read it
Fast inference of deep neural networks in FPGAs for particle physics

Recent results at the Large Hadron Collider (LHC) have pointed to enhanc...
04/16/2018 ∙ by Javier Duarte, et al. ∙ 0 ∙ share

read it
RT-DAP: A Real-Time Data Analytics Platform for Large-scale Industrial Process Monitoring and Control

In most process control systems nowadays, process measurements are perio...
02/22/2018 ∙ by Song Han, et al. ∙ 0 ∙ share

read it
Efficient Sparse-Winograd Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are computationally intensive, whic...
02/18/2018 ∙ by Xingyu Liu, et al. ∙ 0 ∙ share

read it
ADC: Automated Deep Compression and Acceleration with Reinforcement Learning

Model compression is an effective technique facilitating the deployment ...
02/10/2018 ∙ by Yihui He, et al. ∙ 0 ∙ share

read it
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

Large-scale distributed training requires significant communication band...
12/05/2017 ∙ by Yujun Lin, et al. ∙ 0 ∙ share

read it
Deep Generative Adversarial Networks for Compressed Sensing Automates MRI

Magnetic resonance image (MRI) reconstruction is a severely ill-posed li...
05/31/2017 ∙ by Morteza Mardani, et al. ∙ 0 ∙ share

read it
Exploring the Regularity of Sparse Structure in Convolutional Neural Networks

Sparsity helps reduce the computational complexity of deep neural networ...
05/24/2017 ∙ by Huizi Mao, et al. ∙ 0 ∙ share

read it
Classification of Neurological Gait Disorders Using Multi-task Feature Learning

As our population ages, neurological impairments and degeneration of the...
12/08/2016 ∙ by Ioannis Papavasileiou, et al. ∙ 0 ∙ share

read it
ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA

Long Short-Term Memory (LSTM) is widely used in speech recognition. In o...
12/01/2016 ∙ by Song Han, et al. ∙ 0 ∙ share

read it
DSD: Dense-Sparse-Dense Training for Deep Neural Networks

Modern deep neural networks have a large number of parameters, making th...
07/15/2016 ∙ by Song Han, et al. ∙ 0 ∙ share

read it
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size

Recent research on deep neural networks has focused primarily on improvi...
02/24/2016 ∙ by Forrest N. Iandola, et al. ∙ 0 ∙ share

read it
Generate Image Descriptions based on Deep RNN and Memory Cells for Images Features

Generating natural language descriptions for images is a challenging tas...
02/05/2016 ∙ by Shijian Tang, et al. ∙ 0 ∙ share

read it
EIE: Efficient Inference Engine on Compressed Deep Neural Network

State-of-the-art deep neural networks (DNNs) have hundreds of millions o...
02/04/2016 ∙ by Song Han, et al. ∙ 0 ∙ share

read it
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

Neural networks are both computationally intensive and memory intensive,...
10/01/2015 ∙ by Song Han, et al. ∙ 0 ∙ share

read it
Learning both Weights and Connections for Efficient Neural Networks

Neural networks are both computationally intensive and memory intensive,...
06/08/2015 ∙ by Song Han, et al. ∙ 0 ∙ share

read it
Robust Face Recognition using Local Illumination Normalization and Discriminant Feature Point Selection

Face recognition systems must be robust to the variation of various fact...
12/11/2012 ∙ by Song Han, et al. ∙ 0 ∙ share

read it

Song Han

Featured Co-authors