OpTorch: Optimized deep learning architectures for resource limited environments

05/03/2021
by   Salman Ahmed, et al.
0

Deep learning algorithms have made many breakthroughs and have various applications in real life. Computational resources become a bottleneck as the data and complexity of the deep learning pipeline increases. In this paper, we propose optimized deep learning pipelines in multiple aspects of training including time and memory. OpTorch is a machine learning library designed to overcome weaknesses in existing implementations of neural network training. OpTorch provides features to train complex neural networks with limited computational resources. OpTorch achieved the same accuracy as existing libraries on Cifar-10 and Cifar-100 datasets while reducing memory usage to approximately 50 in deep learning pipelines. In our experiments, parallel encoding-decoding along with sequential checkpoints results in much improved memory and time usage while keeping the accuracy similar to existing pipelines. OpTorch python package is available at available at https://github.com/cbrl-nuces/optorch

READ FULL TEXT
research
08/31/2018

Full Workspace Generation of Serial-link Manipulators by Deep Learning based Jacobian Estimation

Apart from solving complicated problems that require a certain level of ...
research
11/01/2022

Efficient AlphaFold2 Training using Parallel Evoformer and Branch Parallelism

The accuracy of AlphaFold2, a frontier end-to-end structure prediction s...
research
02/17/2022

Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines

Preprocessing pipelines in deep learning aim to provide sufficient data ...
research
04/24/2019

Low-Memory Neural Network Training: A Technical Report

Memory is increasingly often the bottleneck when training neural network...
research
08/14/2018

DeepNeuro: an open-source deep learning toolbox for neuroimaging

Translating neural networks from theory to clinical practice has unique ...
research
10/25/2021

Parameter Prediction for Unseen Deep Architectures

Deep learning has been successful in automating the design of features i...
research
07/12/2022

HelixFold: An Efficient Implementation of AlphaFold2 using PaddlePaddle

Accurate protein structure prediction can significantly accelerate the d...

Please sign up or login with your details

Forgot password? Click here to reset