  Deep learning algorithms have made many breakthroughs and have various
applications in real life. Computational resources become a bottleneck as the
data and complexity of the deep learning pipeline increases. In this paper, we
propose optimized deep learning pipelines in multiple aspects of training
including time and memory. OpTorch is a machine learning library designed to
overcome weaknesses in existing implementations of neural network training.
OpTorch provides features to train complex neural networks with limited
computational resources. OpTorch achieved the same accuracy as existing
libraries on Cifar-10 and Cifar-100 datasets while reducing memory usage to
approximately 50%. We also explore the effect of weights on total memory usage
in deep learning pipelines. In our experiments, parallel encoding-decoding
along with sequential checkpoints results in much improved memory and time
usage while keeping the accuracy similar to existing pipelines. OpTorch python
package is available at available at https://github.com/cbrl-nuces/optorch
