  Learning text representation is crucial for text classification and other
language related tasks. There are a diverse set of text representation networks
in the literature, and how to find the optimal one is a non-trivial problem.
Recently, the emerging Neural Architecture Search (NAS) techniques have
demonstrated good potential to solve the problem. Nevertheless, most of the
existing works of NAS focus on the search algorithms and pay little attention
to the search space. In this paper, we argue that the search space is also an
important human prior to the success of NAS in different applications. Thus, we
propose a novel search space tailored for text representation. Through
automatic search, the discovered network architecture outperforms
state-of-the-art models on various public datasets on text classification and
natural language inference tasks. Furthermore, some of the design principles
found in the automatic network agree well with human intuition.
