Wonyong Sung | DeepAI

DeepAI AI Chat

Featured Co-authors

Sungho Shin
21 publications
Chanwoo Kim
21 publications
Jungwook Choi
17 publications
Minsoo Kim
16 publications
Seongmin Park
13 publications
Kyuhong Shim
12 publications
Kyuyeon Hwang
10 publications
Swagath Venkataramani
10 publications
Minjae Lee
10 publications
Yoonho Boo
5 publications
Vijayalakshmi Srinivasan
4 publications

∙ 08/13/2023

Token-Scaled Logit Distillation for Ternary Weight Generative Language Models

Generative Language Models (GLMs) have shown impressive performance in t...

0 Minsoo Kim, et al. ∙

∙ 02/23/2023

Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers

Pre-trained Transformer models such as BERT have shown great success in ...

0 Minsoo Kim, et al. ∙

∙ 01/29/2023

Exploring Attention Map Reuse for Efficient Transformer Neural Networks

Transformer-based deep neural networks have achieved great success in va...

0 Kyuhong Shim, et al. ∙

∙ 12/29/2022

Macro-block dropout for improved regularization in training end-to-end speech recognition models

This paper proposes a new regularization algorithm referred to as macro-...

0 Chanwoo Kim, et al. ∙

∙ 10/01/2022

A Comparison of Transformer, Convolutional, and Recurrent Neural Networks on Phoneme Recognition

Phoneme recognition is a very important part of speech recognition that ...

0 Kyuhong Shim, et al. ∙

∙ 03/19/2022

Similarity and Content-based Phonetic Self Attention for Speech Recognition

Transformer-based speech recognition models have achieved great success ...

0 Kyuhong Shim, et al. ∙

∙ 02/22/2022

Korean Tokenization for Beam Search Rescoring in Speech Recognition

The performance of automatic speech recognition (ASR) models can be grea...

0 Kyuhong Shim, et al. ∙

∙ 10/07/2021

Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling

While Transformer-based models have shown impressive language modeling p...

0 Kyuhong Shim, et al. ∙

∙ 09/30/2020

Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks

The quantization of deep neural networks (QDNNs) has been actively studi...

0 Yoonho Boo, et al. ∙

∙ 09/05/2020

S-SGD: Symmetrical Stochastic Gradient Descent with Weight Noise Injection for Reaching Flat Minima

The stochastic gradient descent (SGD) method is most widely used for dee...

0 Wonyong Sung, et al. ∙

∙ 05/31/2020

Quantized Neural Networks: Characterization and Holistic Optimization

Quantized deep neural networks (QDNNs) are necessary for low-power, high...

0 Yoonho Boo, et al. ∙

∙ 02/02/2020

SQWA: Stochastic Quantized Weight Averaging for Improving the Generalization Capability of Low-Precision Deep Neural Networks

Designing a deep neural network (DNN) with good generalization capabilit...

0 Sungho Shin, et al. ∙

∙ 09/04/2019

Empirical Analysis of Knowledge Distillation Technique for Optimization of Quantized Deep Neural Networks

Knowledge distillation (KD) is a very popular method for model size redu...

0 Sungho Shin, et al. ∙

∙ 11/05/2018

Workload-aware Automatic Parallelization for Multi-GPU DNN Training

Deep neural networks (DNNs) have emerged as successful solutions for var...

1 Sungho Shin, et al. ∙

∙ 03/30/2018

Single Stream Parallelization of Recurrent Neural Networks for Low Power and Fast Inference

As neural network algorithms show high performance in many applications,...

0 Wonyong Sung, et al. ∙

∙ 07/01/2017

Structured Sparse Ternary Weight Coding of Deep Neural Networks for Efficient Hardware Implementations

Deep neural networks (DNNs) usually demand a large amount of operations ...

0 Yoonho Boo, et al. ∙

∙ 11/19/2016

Quantized neural network design under weight capacity constraint

The complexity of deep neural network algorithms for hardware implementa...

0 Sungho Shin, et al. ∙

∙ 10/30/2016

Compact Deep Convolutional Neural Networks With Coarse Pruning

The learning capability of a neural network improves with increasing dep...

0 Sajid Anwar, et al. ∙

∙ 09/13/2016

Character-Level Language Modeling with Hierarchical Recurrent Neural Networks

Recurrent neural network (RNN) based character-level language models (CL...

0 Kyuyeon Hwang, et al. ∙

∙ 08/14/2016

Dynamic Hand Gesture Recognition for Wearable Devices with Low Complexity Recurrent Neural Networks

Gesture recognition is a very essential technology for many wearable dev...

0 Sungho Shin, et al. ∙

∙ 02/04/2016

FPGA Based Implementation of Deep Neural Networks Using On-chip Memory Only

Deep neural networks (DNNs) demand a very large amount of computation an...

0 Jinhwan Park, et al. ∙

∙ 01/25/2016

Character-Level Incremental Speech Recognition with Recurrent Neural Networks

In real-time speech recognition applications, the latency is an importan...

0 Kyuyeon Hwang, et al. ∙

∙ 12/30/2015

Online Keyword Spotting with a Character-Level Recurrent Neural Network

In this paper, we propose a context-aware keyword spotting model employi...

0 Kyuyeon Hwang, et al. ∙

∙ 12/29/2015

Structured Pruning of Deep Convolutional Neural Networks

Real time application of deep learning algorithms is often hindered by h...

0 Sajid Anwar, et al. ∙

∙ 12/04/2015

Fixed-Point Performance Analysis of Recurrent Neural Networks

Recurrent neural networks have shown excellent performance in many appli...

0 Sungho Shin, et al. ∙

∙ 11/21/2015

Online Sequence Training of Recurrent Neural Networks with Connectionist Temporal Classification

Connectionist temporal classification (CTC) based supervised sequence tr...

0 Kyuyeon Hwang, et al. ∙

∙ 11/20/2015

Resiliency of Deep Neural Networks under Quantization

The complexity of deep neural network algorithms for hardware implementa...

0 Wonyong Sung, et al. ∙

∙ 03/10/2015

Single stream parallelization of generalized LSTM-like RNNs on a GPU

Recurrent neural networks (RNNs) have shown outstanding performance on p...

0 Kyuyeon Hwang, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world