  Deep Neural Networks (DNN) have been widely employed in industry to address
various Natural Language Processing (NLP) tasks. However, many engineers find
it a big overhead when they have to choose from multiple frameworks, compare
different types of models, and understand various optimization mechanisms. An
NLP toolkit for DNN models with both generality and flexibility can greatly
improve the productivity of engineers by saving their learning cost and guiding
them to find optimal solutions to their tasks. In this paper, we introduce
NeuronBlocks\footnote{Code: \url{https://github.com/Microsoft/NeuronBlocks}}
\footnote{Demo: \url{https://youtu.be/x6cOpVSZcdo}}, a toolkit encapsulating a
suite of neural network modules as building blocks to construct various DNN
models with complex architecture. This toolkit empowers engineers to build,
train, and test various NLP models through simple configuration of JSON files.
The experiments on several NLP datasets such as GLUE, WikiQA and CoNLL-2003
demonstrate the effectiveness of NeuronBlocks.
