  Machine learning libraries such as TensorFlow and PyTorch simplify model
implementation. However, researchers are still required to perform a
non-trivial amount of manual tasks such as GPU allocation, training status
tracking, and comparison of models with different hyperparameter settings. We
propose a system to handle these tasks and help researchers focus on models. We
present the requirements of the system based on a collection of discussions
from an online study group comprising 25k members. These include automatic GPU
allocation, learning status visualization, handling model parameter snapshots
as well as hyperparameter modification during learning, and comparison of
performance metrics between models via a leaderboard. We describe the system
architecture that fulfills these requirements and present a proof-of-concept
implementation, NAVER Smart Machine Learning (NSML). We test the system and
confirm substantial efficiency improvements for model development.
