research
∙
07/17/2023
Retentive Network: A Successor to Transformer for Large Language Models
In this work, we propose Retentive Network (RetNet) as a foundation arch...
share
research
∙
04/08/2023
FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement
With the increasing data volume, there is a trend of using large-scale p...
research
∙
12/29/2021
Dense-to-Sparse Gate for Mixture-of-Experts
Mixture-of-experts (MoE) is becoming popular due to its success in impro...
research
∙
10/19/2018
Towards Efficient Large-Scale Graph Neural Network Computing
Recent deep learning models have moved beyond low-dimensional regular gr...
research
∙
05/22/2018