b'Lingpeng Kong'

research

∙ 08/09/2023

Extrapolating Large Language Models to Non-English by Aligning Languages

Due to the unbalanced training data distribution, the language ability o...

0 Wenhao Zhu, et al. ∙

research

∙ 07/18/2023

Linearized Relative Positional Encoding

Relative positional encoding is widely used in vanilla and linear transf...

0 Zhen Qin, et al. ∙

research

∙ 06/11/2023

Language Versatilists vs. Specialists: An Empirical Revisiting on Multilingual Transfer Ability

Multilingual transfer ability, which reflects how well the models fine-t...

0 Jiacheng Ye, et al. ∙

research

∙ 06/10/2023

INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation

Neural machine translation has achieved promising results on many transl...

0 Wenhao Zhu, et al. ∙

research

∙ 06/07/2023

M^3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning

Instruction tuning has significantly advanced large language models (LLM...

0 Lei Li, et al. ∙

research

∙ 05/25/2023

Decomposing the Enigma: Subgoal-based Demonstration Learning for Formal Theorem Proving

Large language models (LLMs) present an intriguing avenue of exploration...

0 Xueliang Zhao, et al. ∙

research

∙ 05/23/2023

Can Language Models Understand Physical Concepts?

Language models (LMs) gradually become general-purpose interfaces in the...

0 Lei Li, et al. ∙

research

∙ 05/23/2023

Generating Data for Symbolic Language with Large Language Models

While large language models (LLMs) bring not only performance but also c...

0 Jiacheng Ye, et al. ∙

research

∙ 05/17/2023

Statistical Knowledge Assessment for Generative Language Models

Generative Language Models (GLMs) have demonstrated capabilities to stor...

0 Qingxiu Dong, et al. ∙

research

∙ 05/14/2023

A Cognitive Stimulation Dialogue System with Multi-source Knowledge Fusion for Elders with Cognitive Impairment

When communicating with elders with cognitive impairment, cognitive stim...

0 Jiyue Jiang, et al. ∙

research

∙ 05/08/2023

Toeplitz Neural Network for Sequence Modeling

Sequence modeling has important applications in natural language process...

0 Zhen Qin, et al. ∙

research

∙ 04/18/2023

TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Models

Data augmentation has been established as an efficacious approach to sup...

3 Yuwei Yin, et al. ∙

research

∙ 04/10/2023

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis

Large language models (LLMs) have demonstrated remarkable potential in h...

0 Wenhao Zhu, et al. ∙

research

∙ 03/27/2023

Fine-grained Audible Video Description

We explore a new task for audio-visual-language modeling called fine-gra...

0 Xuyang Shen, et al. ∙

research

∙ 03/07/2023

A Challenging Benchmark for Low-Resource Learning

With promising yet saturated results in high-resource settings, low-reso...

0 Yudong Wang, et al. ∙

research

∙ 02/24/2023

Retrieved Sequence Augmentation for Protein Representation Learning

Protein language models have excelled in a variety of tasks, ranging fro...

0 Chang Ma, et al. ∙

research

∙ 02/11/2023

A Reparameterized Discrete Diffusion Model for Text Generation

This work studies discrete diffusion probabilistic models with applicati...

0 Lin Zheng, et al. ∙

research

∙ 02/11/2023

Compositional Exemplars for In-context Learning

Large pretrained language models (LMs) have shown impressive In-Context ...

0 Jiacheng Ye, et al. ∙

research

∙ 12/20/2022

Lego-MT: Towards Detachable Models in Massively Multilingual Machine Translation

Traditional multilingual neural machine translation (MNMT) uses a single...

0 Fei Yuan, et al. ∙

research

∙ 12/20/2022

Self-adaptive In-context Learning

Despite the surprising few-shot performance of in-context learning (ICL)...

0 Zhiyong Wu, et al. ∙

research

∙ 12/19/2022

Explanation Regeneration via Information Bottleneck

Explaining the black-box predictions of NLP models naturally and accurat...

0 Qintong Li, et al. ∙

research

∙ 10/24/2022

An Empirical Revisiting of Linguistic Knowledge Fusion in Language Understanding Tasks

Though linguistic knowledge emerges during large-scale language model pr...

0 Changlong Yu, et al. ∙

research

∙ 10/22/2022

ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback

Recently, dataset-generation-based zero-shot learning has shown promisin...

0 Jiacheng Ye, et al. ∙

research

∙ 10/19/2022

The Devil in Linear Transformer

Linear transformers aim to reduce the quadratic space-time complexity of...

0 Zhen Qin, et al. ∙

research

∙ 10/17/2022

DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models

Recently, diffusion models have emerged as a new paradigm for generative...

0 Shansan Gong, et al. ∙

research

∙ 10/14/2022

CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling

Transformer has achieved remarkable success in language, image, and spee...

1 Jun Zhang, et al. ∙

research

∙ 06/21/2022

Vicinity Vision Transformer

Vision transformers have shown great success on numerous computer vision...

12 Weixuan Sun, et al. ∙

research

∙ 05/29/2022

CoNT: Contrastive Neural Text Generation

Recently, contrastive learning attracts increasing interests in neural t...

0 Chenxin An, et al. ∙

research

∙ 05/25/2022

ZeroGen^+: Self-Guided High-Quality Data Generation in Efficient Zero-Shot Learning

Nowadays, owing to the superior capacity of the large pre-trained langua...

0 Jiahui Gao, et al. ∙

research

∙ 04/20/2022

Event Transition Planning for Open-ended Text Generation

Open-ended text generation tasks, such as dialogue generation and story ...

2 Qintong Li, et al. ∙

research

∙ 04/10/2022

Linear Complexity Randomized Self-attention Mechanism

Recently, random feature attentions (RFAs) are proposed to approximate t...

0 Lin Zheng, et al. ∙

research

∙ 02/17/2022

cosFormer: Rethinking Softmax in Attention

Transformer has shown great successes in natural language processing, co...

20 Zhen Qin, et al. ∙

research

∙ 02/17/2022

Revisiting Over-smoothing in BERT from the Perspective of Graph

Recently over-smoothing phenomenon of Transformer-based models is observ...

0 Han Shi, et al. ∙

research

∙ 02/16/2022

ZeroGen: Efficient Zero-shot Learning via Dataset Generation

There is a growing interest in dataset generation recently due to the su...

18 Jiacheng Ye, et al. ∙

research

∙ 02/13/2022

A Contrastive Framework for Neural Text Generation

Text generation is of great importance to many natural language processi...

0 Yixuan Su, et al. ∙

research

∙ 01/16/2022

SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samples

Unsupervised sentence embedding aims to obtain the most appropriate embe...

2 Hao Wang, et al. ∙

research

∙ 12/15/2021

Oracle Linguistic Graphs Complement a Pretrained Transformer Language Model: A Cross-formalism Comparison

We examine the extent to which, in principle, linguistic graph represent...

0 Jakob Prange, et al. ∙

research

∙ 10/06/2021

ABC: Attention with Bounded-memory Control

Transformer architectures have achieved state-of-the-art results on a va...

0 Hao Peng, et al. ∙

research

∙ 10/06/2021

Ripple Attention for Visual Perception with Sub-quadratic Complexity

Transformer architectures are now central to modeling in natural languag...

0 Lin Zheng, et al. ∙

research

∙ 05/31/2021

Cascaded Head-colliding Attention

Transformers have advanced the field of natural language processing (NLP...

0 Lin Zheng, et al. ∙

research

∙ 03/03/2021

Random Feature Attention

Transformers are state-of-the-art models for a variety of sequence model...

0 Hao Peng, et al. ∙

research

∙ 02/04/2021

Adaptive Semiparametric Language Models

We present a language model that combines a large parametric neural netw...

0 Dani Yogatama, et al. ∙

research

∙ 05/27/2020

Syntactic Structure Distillation Pretraining For Bidirectional Encoders

Textual representation learners trained on large amounts of data have ac...

0 Adhiguna Kuncoro, et al. ∙

research

∙ 10/18/2019

A Mutual Information Maximization Perspective of Language Representation Learning

We show state-of-the-art word representation learning methods maximize a...

0 Lingpeng Kong, et al. ∙

research

∙ 10/01/2019

Putting Machine Translation in Context with the Noisy Channel Model

We show that Bayes' rule provides a compelling mechanism for controlling...

0 Lei Yu, et al. ∙

research

∙ 06/03/2019

Episodic Memory in Lifelong Language Learning

We introduce a lifelong language learning setup where a model needs to l...

8 Cyprien de Masson d'Autume, et al. ∙

research

∙ 01/31/2019

Learning and Evaluating General Linguistic Intelligence

We define general linguistic intelligence as the ability to reuse previo...

8 Dani Yogatama, et al. ∙

research

∙ 01/27/2019

Variational Smoothing in Recurrent Neural Network Language Models

We present a new theoretical perspective of data noising in recurrent ne...

0 Lingpeng Kong, et al. ∙

research

∙ 11/26/2018

Sentence Encoding with Tree-constrained Relation Networks

The meaning of a sentence is a function of the relations that hold betwe...

0 Lei Yu, et al. ∙

research

∙ 11/06/2018

Neural Phrase-to-Phrase Machine Translation

In this paper, we propose Neural Phrase-to-Phrase Machine Translation (N...

0 Jiangtao Feng, et al. ∙

Lingpeng Kong

Featured Co-authors