Exploring Low-Cost Transformer Model Compression for Large-Scale Commercial Reply Suggestions

11/27/2021
by   Vaishnavi Shrivastava, et al.
5

Fine-tuning pre-trained language models improves the quality of commercial reply suggestion systems, but at the cost of unsustainable training times. Popular training time reduction approaches are resource intensive, thus we explore low-cost model compression techniques like Layer Dropping and Layer Freezing. We demonstrate the efficacy of these techniques in large-data scenarios, enabling the training time reduction for a commercial email reply suggestion system by 42 engagement. We further study the robustness of these techniques to pre-trained model and dataset size ablation, and share several insights and recommendations for commercial applications.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset