research
∙
09/19/2023
AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models
Audio-visual representation learning aims to develop systems with human-...
share
research
∙
09/18/2023
Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech
Text language models have shown remarkable zero-shot capability in gener...
research
∙
06/10/2023
Improving Non-autoregressive Translation Quality with Pretrained Language Model, Embedding Distillation and Upsampling Strategy for CTC
Non-autoregressive approaches aim to improve the inference speed of tran...
research
∙
06/08/2023
Revealing the Blind Spot of Sentence Encoder Evaluation by HEROS
Existing sentence textual similarity benchmark datasets only use a singl...
research
∙
06/03/2023
SpeechGen: Unlocking the Generative Power of Speech Language Models with Prompts
Large language models (LLMs) have gained considerable attention for Arti...
research
∙
06/01/2023
How to Estimate Model Transferability of Pre-Trained Speech Models?
In this work, we introduce a “score-based assessment” framework for esti...
research
∙
05/30/2023
MiniSUPERB: Lightweight Benchmark for Self-supervised Speech Models
Self-supervised learning (SSL) is a popular research topic in speech pro...
research
∙
05/29/2023
Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target
Spoken Language Understanding (SLU) is a task that aims to extract seman...
research
∙
05/22/2023
The defender's perspective on automatic speaker verification: An overview
Automatic speaker verification (ASV) plays a critical role in security-s...
research
∙
05/18/2023
ML-SUPERB: Multilingual Speech Universal PERformance Benchmark
Speech processing Universal PERformance Benchmark (SUPERB) is a leaderbo...
research
∙
05/12/2023
Improving Cascaded Unsupervised Speech Translation with Denoising Back-translation
Most of the speech translation models heavily rely on parallel data, whi...
research
∙
05/03/2023
Can Large Language Models Be an Alternative to Human Evaluations?
Human evaluation is indispensable and inevitable for assessing the quali...
research
∙
03/21/2023
Personalized Lightweight Text-to-Speech: Voice Cloning with Adaptive Structured Pruning
Personalized TTS is an exciting and highly desired application that allo...
research
∙
03/15/2023
Cascading and Direct Approaches to Unsupervised Constituency Parsing on Spoken Sentences
Past work on unsupervised parsing is constrained to written form. In thi...
research
∙
03/01/2023
SpeechPrompt v2: Prompt Tuning for Speech Classification Tasks
Prompt tuning is a technology that tunes a small set of parameters to st...
research
∙
02/24/2023
Ensemble knowledge distillation of self-supervised speech models
Distilled self-supervised models have shown competitive performance and ...
research
∙
02/12/2023
Position Matters! Empirical Study of Order Effect in Knowledge-grounded Dialogue
With the power of large pretrained language models, various research wor...
research
∙
01/30/2023
Hierarchical Programmatic Reinforcement Learning via Learning to Compose Programs
Aiming to produce reinforcement learning (RL) policies that are human-in...
research
∙
12/20/2022
SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks
Spoken language understanding (SLU) tasks have been studied for many dec...
research
∙
12/02/2022
General Framework for Self-Supervised Model Priming for Parameter-Efficient Fine-tuning
Parameter-efficient methods (like Prompt or Adapters) for adapting pre-t...
research
∙
12/01/2022
CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Models
Self-supervised learning (SSL) is a powerful technique for learning repr...
research
∙
11/30/2022
EURO: ESPnet Unsupervised ASR Open-source Toolkit
This paper describes the ESPnet Unsupervised ASR Open-source Toolkit (EU...
research
∙
11/29/2022
Model Extraction Attack against Self-supervised Speech Models
Self-supervised learning (SSL) speech models generate meaningful represe...
research
∙
11/17/2022
Compressing Transformer-based self-supervised models for speech processing
Despite the success of Transformers in self-supervised learning with app...
research
∙
11/17/2022
MelHuBERT: A simplified HuBERT on Mel spectrogram
Self-supervised models have had great success in learning speech represe...
research
∙
11/15/2022
Introducing Semantics into Speech Encoders
Recent studies find existing self-supervised speech encoders contain pri...
research
∙
11/06/2022
Bridging Speech and Textual Pre-trained Models with Unsupervised ASR
Spoken language understanding (SLU) is a task aiming to extract high-lev...
research
∙
11/04/2022
Once-for-All Sequence Compression for Self-Supervised Speech Models
The sequence length along the time axis is often the dominant factor of ...
research
∙
11/02/2022
M-SpeechCLIP: Leveraging Large-Scale, Pre-Trained Models for Multilingual Speech to Image Retrieval
This work investigates the use of large-scale, pre-trained models (CLIP ...
research
∙
11/01/2022
T5lephone: Bridging Speech and Text Self-supervised Models for Spoken Language Understanding via Phoneme level T5
In Spoken language understanding (SLU), a natural solution is concatenat...
research
∙
10/27/2022
Multimodal Transformer Distillation for Audio-Visual Synchronization
Audio-visual synchronization aims to determine whether the mouth movemen...
research
∙
10/16/2022
SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning
We present the SUPERB challenge at SLT 2022, which aims at learning self...
research
∙
10/14/2022
Improving generalizability of distilled self-supervised speech processing models under distorted settings
Self-supervised learned (SSL) speech pre-trained models perform well acr...
research
∙
10/13/2022
On Compressing Sequences for Self-Supervised Speech Models
Compressing self-supervised models has become increasingly necessary, as...
research
∙
10/13/2022
On the Utility of Self-supervised Models for Prosody-related Tasks
Self-Supervised Learning (SSL) from speech data has produced models that...
research
∙
10/10/2022
Exploring Efficient-tuning Methods in Self-supervised Speech Models
In this study, we aim to explore efficient tuning methods for speech sel...
research
∙
10/06/2022
How Far Are We from Real Synonym Substitution Attacks?
In this paper, we explore the following question: how far are we from re...
research
∙
10/03/2022
Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection
Audio-visual active speaker detection (AVASD) is well-developed, and now...
research
∙
10/03/2022
SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model
Data-driven speech processing models usually perform well with a large a...
research
∙
09/26/2022
The Ability of Self-Supervised Speech Models for Audio Representations
Self-supervised learning (SSL) speech models have achieved unprecedented...
research
∙
06/27/2022
Few-Shot Cross-Lingual TTS Using Transferable Phoneme Embedding
This paper studies a transferable phoneme embedding framework that aims ...
research
∙
06/08/2022
Few-shot Prompting Towards Controllable Response Generation
Much literature has shown that prompt-based learning is an efficient met...
research
∙
05/30/2022
Searching for the Essence of Adversarial Perturbations
Neural networks have achieved the state-of-the-art performance in variou...
research
∙
05/24/2022
Structured Prompt Tuning
We propose structured prompt tuning, a simple and effective method to im...
research
∙
05/21/2022
Self-Supervised Speech Representation Learning: A Review
Although supervised deep learning has revolutionized speech and audio pr...
research
∙
05/03/2022
Meta Learning for Natural Language Processing: A Survey
Deep learning has been the mainstream technique in natural language proc...
research
∙
04/30/2022
AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks
Transformer-based pre-trained models with millions of parameters require...
research
∙
04/15/2022
XDBERT: Distilling Visual Information to BERT from Cross-Modal Systems to Improve Language Understanding
Transformer-based models are widely used in natural language understandi...
research
∙
04/10/2022
Re-Examining Human Annotations for Interpretable NLP
Explanation methods in Interpretable NLP often explain the model's decis...
research
∙
04/09/2022