research
∙
05/25/2023
Landmark Attention: Random-Access Infinite Context Length for Transformers
While transformers have shown remarkable success in natural language pro...
share
research
∙
02/07/2023
Learning Translation Quality Evaluation on Low Resource Languages from Large Language Models
Learned metrics such as BLEURT have in recent years become widely employ...
research
∙
05/30/2022
On Avoiding Local Minima Using Gradient Descent With Large Learning Rates
It has been widely observed in training of neural networks that when app...
research
∙
02/03/2022
Characterizing Finding Good Data Orderings for Fast Convergence of Sequential Gradient Methods
While SGD, which samples from the data with replacement is widely studie...
research
∙
06/16/2021
Simultaneous Training of Partially Masked Neural Networks
For deploying deep learning models to lower end devices, it is necessary...
research
∙
03/03/2021
Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates
It has been experimentally observed that the efficiency of distributed t...
research
∙
08/03/2020