research
∙
08/01/2023
CodeBPE: Investigating Subtokenization Options for Large Language Model Pretraining on Source Code
Recent works have widely adopted large language model pretraining for so...
share
research
∙
06/30/2023
Should you marginalize over possible tokenizations?
Autoregressive language models (LMs) map token sequences to probabilitie...
research
∙
12/12/2022
Parameter-Efficient Finetuning of Transformers for Source Code
Pretrained Transformers achieve state-of-the-art performance in various ...
research
∙
02/16/2022
Probing Pretrained Models of Source Code
Deep learning models are widely used for solving challenging code proces...
research
∙
12/29/2021
Machine Learning Methods for Spectral Efficiency Prediction in Massive MIMO Systems
Channel decoding, channel detection, channel assessment, and resource ma...
research
∙
07/21/2021
On the Memorization Properties of Contrastive Learning
Memorization studies of deep neural networks (DNNs) help to understand w...
research
∙
06/29/2021
On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay
Despite the conventional wisdom that using batch normalization with weig...
research
∙
10/23/2020
Neural Code Completion with Anonymized Variable Names
Source code processing heavily relies on the methods widely used in natu...
research
∙
10/23/2020
A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code
There is an emerging interest in the application of deep learning models...
research
∙
10/15/2020
Empirical Study of Transformers for Source Code
Initially developed for natural language processing (NLP), Transformers ...
research
∙
07/16/2020
On Power Laws in Deep Ensembles
Ensembles of deep neural networks are known to achieve state-of-the-art ...
research
∙
05/14/2020
Deep Ensembles on a Fixed Memory Budget: One Wide Network or Several Thinner Ones?
One of the generally accepted views of modern deep learning is that incr...
research
∙
11/13/2019
Structured Sparsification of Gated Recurrent Neural Networks
Recently, a lot of techniques were developed to sparsify the weights of ...
research
∙
12/12/2018
Bayesian Sparsification of Gated Recurrent Neural Networks
Bayesian methods have been successfully applied to sparsify weights of n...
research
∙
10/25/2018
Bayesian Compression for Natural Language Processing
In natural language processing, a lot of the tasks are successfully solv...
research
∙
07/31/2017