  Online forms are widely used to collect data from human and have a
multi-billion market. Many software products provide online services for
creating semi-structured forms where questions and descriptions are organized
by pre-defined structures. However, the design and creation process of forms is
still tedious and requires expert knowledge. To assist form designers, in this
work we present FormLM to model online forms (by enhancing pre-trained language
model with form structural information) and recommend form creation ideas
(including question / options recommendations and block type suggestion). For
model training and evaluation, we collect the first public online form dataset
with 62K online forms. Experiment results show that FormLM significantly
outperforms general-purpose language models on all tasks, with an improvement
by 4.71 on Question Recommendation and 10.6 on Block Type Suggestion in terms
of ROUGE-1 and Macro-F1, respectively.
