b'Hung-Yi Lee'

research

∙ 09/19/2023

AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models

Audio-visual representation learning aims to develop systems with human-...

0 Yuan Tseng, et al. ∙

research

∙ 09/18/2023

Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech

Text language models have shown remarkable zero-shot capability in gener...

0 Chien-yu Huang, et al. ∙

research

∙ 06/10/2023

Improving Non-autoregressive Translation Quality with Pretrained Language Model, Embedding Distillation and Upsampling Strategy for CTC

Non-autoregressive approaches aim to improve the inference speed of tran...

0 Shen-sian Syu, et al. ∙

research

∙ 06/08/2023

Revealing the Blind Spot of Sentence Encoder Evaluation by HEROS

Existing sentence textual similarity benchmark datasets only use a singl...

0 Cheng-Han Chiang, et al. ∙

research

∙ 06/03/2023

SpeechGen: Unlocking the Generative Power of Speech Language Models with Prompts

Large language models (LLMs) have gained considerable attention for Arti...

0 Haibin Wu, et al. ∙

research

∙ 06/01/2023

How to Estimate Model Transferability of Pre-Trained Speech Models?

In this work, we introduce a “score-based assessment” framework for esti...

0 Zih-Ching Chen, et al. ∙

research

∙ 05/30/2023

MiniSUPERB: Lightweight Benchmark for Self-supervised Speech Models

Self-supervised learning (SSL) is a popular research topic in speech pro...

0 Yu-Hsiang Wang, et al. ∙

research

∙ 05/29/2023

Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target

Spoken Language Understanding (SLU) is a task that aims to extract seman...

0 Guan-Wei Wu, et al. ∙

research

∙ 05/22/2023

The defender's perspective on automatic speaker verification: An overview

Automatic speaker verification (ASV) plays a critical role in security-s...

0 Haibin Wu, et al. ∙

research

∙ 05/18/2023

ML-SUPERB: Multilingual Speech Universal PERformance Benchmark

Speech processing Universal PERformance Benchmark (SUPERB) is a leaderbo...

0 Jiatong Shi, et al. ∙

research

∙ 05/12/2023

Improving Cascaded Unsupervised Speech Translation with Denoising Back-translation

Most of the speech translation models heavily rely on parallel data, whi...

0 Yu-Kuan Fu, et al. ∙

research

∙ 05/03/2023

Can Large Language Models Be an Alternative to Human Evaluations?

Human evaluation is indispensable and inevitable for assessing the quali...

0 Cheng-Han Chiang, et al. ∙

research

∙ 03/21/2023

Personalized Lightweight Text-to-Speech: Voice Cloning with Adaptive Structured Pruning

Personalized TTS is an exciting and highly desired application that allo...

0 Sung-Feng Huang, et al. ∙

research

∙ 03/15/2023

Cascading and Direct Approaches to Unsupervised Constituency Parsing on Spoken Sentences

Past work on unsupervised parsing is constrained to written form. In thi...

0 Yuan Tseng, et al. ∙

research

∙ 03/01/2023

SpeechPrompt v2: Prompt Tuning for Speech Classification Tasks

Prompt tuning is a technology that tunes a small set of parameters to st...

0 Kai-Wei Chang, et al. ∙

research

∙ 02/24/2023

Ensemble knowledge distillation of self-supervised speech models

Distilled self-supervised models have shown competitive performance and ...

0 Kuan-Po Huang, et al. ∙

research

∙ 02/12/2023

Position Matters! Empirical Study of Order Effect in Knowledge-grounded Dialogue

With the power of large pretrained language models, various research wor...

0 Hsuan Su, et al. ∙

research

∙ 01/30/2023

Hierarchical Programmatic Reinforcement Learning via Learning to Compose Programs

Aiming to produce reinforcement learning (RL) policies that are human-in...

0 Guan-Ting Liu, et al. ∙

research

∙ 12/20/2022

SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks

Spoken language understanding (SLU) tasks have been studied for many dec...

0 Suwon Shon, et al. ∙

research

∙ 12/02/2022

General Framework for Self-Supervised Model Priming for Parameter-Efficient Fine-tuning

Parameter-efficient methods (like Prompt or Adapters) for adapting pre-t...

0 Shih-Cheng Huang, et al. ∙

research

∙ 12/01/2022

CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Models

Self-supervised learning (SSL) is a powerful technique for learning repr...

0 Zih-Ching Chen, et al. ∙

research

∙ 11/30/2022

EURO: ESPnet Unsupervised ASR Open-source Toolkit

This paper describes the ESPnet Unsupervised ASR Open-source Toolkit (EU...

0 Dongji Gao, et al. ∙

research

∙ 11/29/2022

Model Extraction Attack against Self-supervised Speech Models

Self-supervised learning (SSL) speech models generate meaningful represe...

0 Tsu-Yuan Hsu, et al. ∙

research

∙ 11/17/2022

Compressing Transformer-based self-supervised models for speech processing

Despite the success of Transformers in self-supervised learning with app...

0 Tzu-Quan Lin, et al. ∙

research

∙ 11/17/2022

MelHuBERT: A simplified HuBERT on Mel spectrogram

Self-supervised models have had great success in learning speech represe...

0 Tzu-Quan Lin, et al. ∙

research

∙ 11/15/2022

Introducing Semantics into Speech Encoders

Recent studies find existing self-supervised speech encoders contain pri...

10 Derek Xu, et al. ∙

research

∙ 11/06/2022

Bridging Speech and Textual Pre-trained Models with Unsupervised ASR

Spoken language understanding (SLU) is a task aiming to extract high-lev...

0 Jiatong Shi, et al. ∙

research

∙ 11/04/2022

Once-for-All Sequence Compression for Self-Supervised Speech Models

The sequence length along the time axis is often the dominant factor of ...

0 Hsuan-Jui Chen, et al. ∙

research

∙ 11/02/2022

M-SpeechCLIP: Leveraging Large-Scale, Pre-Trained Models for Multilingual Speech to Image Retrieval

This work investigates the use of large-scale, pre-trained models (CLIP ...

0 Layne Berry, et al. ∙

research

∙ 11/01/2022

T5lephone: Bridging Speech and Text Self-supervised Models for Spoken Language Understanding via Phoneme level T5

In Spoken language understanding (SLU), a natural solution is concatenat...

0 Chan-Jan Hsu, et al. ∙

research

∙ 10/27/2022

Multimodal Transformer Distillation for Audio-Visual Synchronization

Audio-visual synchronization aims to determine whether the mouth movemen...

0 Xuanjun Chen, et al. ∙

research

∙ 10/16/2022

SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning

We present the SUPERB challenge at SLT 2022, which aims at learning self...

0 Tzu-hsun Feng, et al. ∙

research

∙ 10/14/2022

Improving generalizability of distilled self-supervised speech processing models under distorted settings

Self-supervised learned (SSL) speech pre-trained models perform well acr...

0 Kuan-Po Huang, et al. ∙

research

∙ 10/13/2022

On Compressing Sequences for Self-Supervised Speech Models

Compressing self-supervised models has become increasingly necessary, as...

0 Yen Meng, et al. ∙

research

∙ 10/13/2022

On the Utility of Self-supervised Models for Prosody-related Tasks

Self-Supervised Learning (SSL) from speech data has produced models that...

0 Guan-Ting Lin, et al. ∙

research

∙ 10/10/2022

Exploring Efficient-tuning Methods in Self-supervised Speech Models

In this study, we aim to explore efficient tuning methods for speech sel...

0 Zih-Ching Chen, et al. ∙

research

∙ 10/06/2022

How Far Are We from Real Synonym Substitution Attacks?

In this paper, we explore the following question: how far are we from re...

0 Cheng-Han Chiang, et al. ∙

research

∙ 10/03/2022

Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection

Audio-visual active speaker detection (AVASD) is well-developed, and now...

0 Xuanjun Chen, et al. ∙

research

∙ 10/03/2022

SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model

Data-driven speech processing models usually perform well with a large a...

0 Yi-Jen Shih, et al. ∙

research

∙ 09/26/2022

The Ability of Self-Supervised Speech Models for Audio Representations

Self-supervised learning (SSL) speech models have achieved unprecedented...

0 Tung-yu Wu, et al. ∙

research

∙ 06/27/2022

Few-Shot Cross-Lingual TTS Using Transferable Phoneme Embedding

This paper studies a transferable phoneme embedding framework that aims ...

0 Wei-Ping Huang, et al. ∙

research

∙ 06/08/2022

Few-shot Prompting Towards Controllable Response Generation

Much literature has shown that prompt-based learning is an efficient met...

0 Hsuan Su, et al. ∙

research

∙ 05/30/2022

Searching for the Essence of Adversarial Perturbations

Neural networks have achieved the state-of-the-art performance in variou...

1 Dennis Y. Menn, et al. ∙

research

∙ 05/24/2022

Structured Prompt Tuning

We propose structured prompt tuning, a simple and effective method to im...

2 Chi-Liang Liu, et al. ∙

research

∙ 05/21/2022

Self-Supervised Speech Representation Learning: A Review

Although supervised deep learning has revolutionized speech and audio pr...

0 Abdelrahman Mohamed, et al. ∙

research

∙ 05/03/2022

Meta Learning for Natural Language Processing: A Survey

Deep learning has been the mainstream technique in natural language proc...

0 Hung-Yi Lee, et al. ∙

research

∙ 04/30/2022

AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks

Transformer-based pre-trained models with millions of parameters require...

0 Chin-Lun Fu, et al. ∙

research

∙ 04/15/2022

XDBERT: Distilling Visual Information to BERT from Cross-Modal Systems to Improve Language Understanding

Transformer-based models are widely used in natural language understandi...

0 Chan-Jan Hsu, et al. ∙

research

∙ 04/10/2022

Re-Examining Human Annotations for Interpretable NLP

Explanation methods in Interpretable NLP often explain the model's decis...

0 Cheng-Han Chiang, et al. ∙

research

∙ 04/09/2022

Understanding, Detecting, and Separating Out-of-Distribution Samples and Adversarial Samples in Text Classification

In this paper, we study the differences and commonalities between statis...

0 Cheng-Han Chiang, et al. ∙

Hung-Yi Lee

Featured Co-authors